 |
|
| Linux Forum Index » Linux Networking » A good free backup progie like Rsync... |
|
Page 2 of 3 Goto page Previous 1, 2, 3 Next |
|
| Author |
Message |
| Grant... |
Posted: Tue Oct 27, 2009 9:57 pm |
|
|
|
Guest
|
On Wed, 28 Oct 2009 03:39:59 +0000 (UTC), terryc <newsninespam-spam at (no spam) woa.com.au> wrote:
Quote: On Mon, 26 Oct 2009 20:52:26 +0100, David Brown wrote:
Gabriel Knight wrote:
Hi all I need a free program to backup a ubuntu server for my school
class, it has to be as good or better than Rdiff and Rsync the server
will use SSH, MYSql and be a file and web server and do a couple of
other things. I need it to be either a gui or text box program.
As others have said, the obvious choice here would be ... rsync.
*sync isn't a abckup system. It is just a copy system.
And a backup is not a copy?
Grant.
--
http://bugsplatter.id.au |
|
|
| Back to top |
|
|
|
| Joe... |
Posted: Tue Oct 27, 2009 10:06 pm |
|
|
|
Guest
|
On 2009-10-28, terryc <newsninespam-spam at (no spam) woa.com.au> wrote:
Quote: On Mon, 26 Oct 2009 20:52:26 +0100, David Brown wrote:
Gabriel Knight wrote:
Hi all I need a free program to backup a ubuntu server for my school
class, it has to be as good or better than Rdiff and Rsync the server
will use SSH, MYSql and be a file and web server and do a couple of
other things. I need it to be either a gui or text box program.
As others have said, the obvious choice here would be ... rsync.
*sync isn't a abckup system. It is just a copy system.
A copy *IS* a backup.
--
Joe - Linux User #449481/Ubuntu User #19733
joe at hits - buffalo dot com
"Hate is baggage, life is too short to go around pissed off all the
time..." - Danny, American History X |
|
|
| Back to top |
|
|
|
| Keith Keller... |
Posted: Tue Oct 27, 2009 10:48 pm |
|
|
|
Guest
|
On 2009-10-28, terryc <newsninespam-spam at (no spam) woa.com.au> wrote:
Quote: On Wed, 28 Oct 2009 14:57:55 +1100, Grant wrote:
*sync isn't a abckup system. It is just a copy system.
And a backup is not a copy?
A backup system has multiple copies, with historical content.
How do you come to that definition? A backup system has as many copies
as desired/needed by business practice. If one copy without history is
sufficient, then that's a good backup.
--keith
--
kkeller-usenet at (no spam) wombat.san-francisco.ca.us
(try just my userid to email me)
AOLSFAQ=http://www.therockgarden.ca/aolsfaq.txt
see X- headers for PGP signature information |
|
|
| Back to top |
|
|
|
| jjg... |
Posted: Wed Oct 28, 2009 12:25 am |
|
|
|
Guest
|
terryc wrote:
Quote: On Wed, 28 Oct 2009 14:57:55 +1100, Grant wrote:
*sync isn't a abckup system. It is just a copy system.
And a backup is not a copy?
A backup system has multiple copies, with historical content.
Well, you _can_ do that with rsync. see e.g. Rubel's article, in
http://www.mikerubel.org/computers/rsync_snapshots/
But, strictly you are right; rsync is not, by itself, a backup system. |
|
|
| Back to top |
|
|
|
| terryc... |
Posted: Wed Oct 28, 2009 12:57 am |
|
|
|
Guest
|
On Tue, 27 Oct 2009 21:48:21 -0700, Keith Keller wrote:
Quote: On 2009-10-28, terryc <newsninespam-spam at (no spam) woa.com.au> wrote:
On Wed, 28 Oct 2009 14:57:55 +1100, Grant wrote:
*sync isn't a abckup system. It is just a copy system.
And a backup is not a copy?
A backup system has multiple copies, with historical content.
How do you come to that definition? A backup system has as many copies
as desired/needed by business practice. If one copy without history is
sufficient, then that's a good backup.
The basic reason is that problems can happen with "the copy". No matter
what your system, one copy is no copy at certain times.
As others have pointed out, it is the system, not the tools (tar, cpio,
rsync, etc).
As you say, it is about what he really needs.
My 2c is that depending on amount of data and frequency of changing, he
could probably just dump his databases and just burn a DVD each night.
$AUS50 for a dvd burner and AUS50c/DVD. Even cheap DVD should last a few
years before they are toasters. |
|
|
| Back to top |
|
|
|
| Joe... |
Posted: Wed Oct 28, 2009 2:27 am |
|
|
|
Guest
|
On 2009-10-28, terryc <newsninespam-spam at (no spam) woa.com.au> wrote:
Quote: On Wed, 28 Oct 2009 14:57:55 +1100, Grant wrote:
*sync isn't a abckup system. It is just a copy system.
And a backup is not a copy?
A backup system has multiple copies, with historical content.
So does rsync, if you use it that way. Rsnapshot is rsync, and I can
give you a restore from my backup drive from any day in the last week,
any week in the last month, and any month in the last year. GFS.
Rsync, on it's own, can do the same, but takes a little more leg work.
--
Joe - Linux User #449481/Ubuntu User #19733
joe at hits - buffalo dot com
"Hate is baggage, life is too short to go around pissed off all the
time..." - Danny, American History X |
|
|
| Back to top |
|
|
|
| terryc... |
Posted: Wed Oct 28, 2009 3:18 am |
|
|
|
Guest
|
On Wed, 28 Oct 2009 12:34:58 +0100, David Brown wrote:
Quote: Since it was going to take days to make the
new copy, and they couldn't be bothered waiting, they cancelled the copy
and did the hardware upgrade anyway. When that failed, everything was
gone.
I actually worked, briefly, for a company that would not buy new backup
tapes. After weeks of requesting new tapes, I eventually worked into the
EDP mangers office and dumped a backup report that took up half a box of
paper, normally 50 pages, and said "It is my belief that if we needed to
recover from a backup, the company could not do it". Then I held up a
sample of the 2nd hand tapes they were so proud of obtaining and left
tape coating over the managers desk. { .
Quote:
As you say, having a single copy is not a backup system by itself.
And even multiple copies on multiple devices at multiple locations is
not a backup system - you need a tried and tested recovery procedure to
make it a backup system.
First thing you check when you become responsible for the company
backups. I've also come across a couple of those in my time. This isn't
too bad if you have a good GFS system and the paper trail (or transaction
logs) still exists.
Quote:
As others have pointed out, it is the system, not the tools (tar, cpio,
rsync, etc).
However, the tools form the central part of the backup system, and
determine the main features you have available. Thus it does make sense
to talk about an rsync backup system.
Between scripts running tar, cpio and diff, there hasn't been much else
in many "branded" backup systems I've used. The part that makes them easy
to use is the database, query and file selection facilities.
The only other trick you need to know is that many put multiple jobs onto
one tape so you need to use /dev/nst0 (no rewind) and move pass the end
of tape(?) marker to be able to manually recover the contents of some of
these tapes. |
|
|
| Back to top |
|
|
|
| David Brown... |
Posted: Wed Oct 28, 2009 3:56 am |
|
|
|
Guest
|
terryc wrote:
Quote: On Mon, 26 Oct 2009 20:52:26 +0100, David Brown wrote:
Gabriel Knight wrote:
Hi all I need a free program to backup a ubuntu server for my school
class, it has to be as good or better than Rdiff and Rsync the server
will use SSH, MYSql and be a file and web server and do a couple of
other things. I need it to be either a gui or text box program.
As others have said, the obvious choice here would be ... rsync.
*sync isn't a abckup system. It is just a copy system.
A backup system means having copies of your data along with a way to
restore it. You are right that rsync on its own is not a backup system,
but it can form the backbone of a backup system. If you use it
appropriately (either with your own scripts or command line arguments,
or using a ready-made tool such as rsnapshot or dirvish), you then have
a backup system. |
|
|
| Back to top |
|
|
|
| David Brown... |
Posted: Wed Oct 28, 2009 5:34 am |
|
|
|
Guest
|
terryc wrote:
Quote: On Tue, 27 Oct 2009 21:48:21 -0700, Keith Keller wrote:
On 2009-10-28, terryc <newsninespam-spam at (no spam) woa.com.au> wrote:
On Wed, 28 Oct 2009 14:57:55 +1100, Grant wrote:
*sync isn't a abckup system. It is just a copy system.
And a backup is not a copy?
A backup system has multiple copies, with historical content.
How do you come to that definition? A backup system has as many copies
as desired/needed by business practice. If one copy without history is
sufficient, then that's a good backup.
The basic reason is that problems can happen with "the copy". No matter
what your system, one copy is no copy at certain times.
For an example, consider what happened to a very large software company
recently when trying to do a hardware upgrade of the storage system.
There was only one copy of the data (an old copy at that). Before doing
the hardware upgrade, they wanted to make a new copy - but there was not
enough space on the backup system. So they deleted their only copy, and
started making a new copy. Since it was going to take days to make the
new copy, and they couldn't be bothered waiting, they cancelled the copy
and did the hardware upgrade anyway. When that failed, everything was gone.
As you say, having a single copy is not a backup system by itself.
And even multiple copies on multiple devices at multiple locations is
not a backup system - you need a tried and tested recovery procedure to
make it a backup system.
Quote: As others have pointed out, it is the system, not the tools (tar, cpio,
rsync, etc).
However, the tools form the central part of the backup system, and
determine the main features you have available. Thus it does make sense
to talk about an rsync backup system.
Quote: As you say, it is about what he really needs.
My 2c is that depending on amount of data and frequency of changing, he
could probably just dump his databases and just burn a DVD each night.
$AUS50 for a dvd burner and AUS50c/DVD. Even cheap DVD should last a few
years before they are toasters. |
|
|
| Back to top |
|
|
|
| David Brown... |
Posted: Wed Oct 28, 2009 7:53 am |
|
|
|
Guest
|
terryc wrote:
Quote: On Wed, 28 Oct 2009 12:34:58 +0100, David Brown wrote:
Since it was going to take days to make the
new copy, and they couldn't be bothered waiting, they cancelled the copy
and did the hardware upgrade anyway. When that failed, everything was
gone.
In case anyone didn't spot the reference:
<http://www.hiptop3.com/archives/sidekick-backup-problems-blamed-on-management>
Quote: I actually worked, briefly, for a company that would not buy new backup
tapes. After weeks of requesting new tapes, I eventually worked into the
EDP mangers office and dumped a backup report that took up half a box of
paper, normally 50 pages, and said "It is my belief that if we needed to
recover from a backup, the company could not do it". Then I held up a
sample of the 2nd hand tapes they were so proud of obtaining and left
tape coating over the managers desk. {  .
I've heard of that kind of thing many times - people often make the
mistake of equating "tape" with "good backup system", regardless of the
procedures used for the backup. And since backup tapes cost money, it's
easy to think that you can save money by buying fewer tapes without
thinking through the consequences.
Quote: As you say, having a single copy is not a backup system by itself.
And even multiple copies on multiple devices at multiple locations is
not a backup system - you need a tried and tested recovery procedure to
make it a backup system.
First thing you check when you become responsible for the company
backups. I've also come across a couple of those in my time. This isn't
too bad if you have a good GFS system and the paper trail (or transaction
logs) still exists.
You don't necessarily need something like a GFS. It's often okay for a
recovery procedure to be somewhat slow and inconvenient, as long as it
works reliably, and within the required timeframe.
But while /you/ may check the recovery procedure, a great many people do
not - that's why I emphasis it so much each time the topic comes up.
Again, it's often a fault with tape-based backups. People think they've
got a great system with incremental and differential backups, but when
disaster strikes they have huge troubles. Maybe they find that they
need to feed in a week's worth of differential backup tapes just to
restore a single file. Or they find that when the hardware failure /
fire / breakin put their server and backup machine out of commission,
they can't read the tapes on a new machine. Or perhaps their backup
software won't let them restore part of their data - so they can't
restore part of yesterday's backup without destroying today's good data.
There are many ways for restores to fail even when the data is
technically safe, but people often don't think through and test their
recovery procedures.
Quote: As others have pointed out, it is the system, not the tools (tar, cpio,
rsync, etc).
However, the tools form the central part of the backup system, and
determine the main features you have available. Thus it does make sense
to talk about an rsync backup system.
Between scripts running tar, cpio and diff, there hasn't been much else
in many "branded" backup systems I've used. The part that makes them easy
to use is the database, query and file selection facilities.
I find rsync particularly useful for doing offsite backups over the
Internet, since it minimises the traffic.
Quote: The only other trick you need to know is that many put multiple jobs onto
one tape so you need to use /dev/nst0 (no rewind) and move pass the end
of tape(?) marker to be able to manually recover the contents of some of
these tapes.
My experience with tape-based backups is that it involved too much
manual work (changing tapes), recovery was slow and awkward, it was very
difficult to be sure of the reliability of the system, and we had no
practical (or cheap) way to test recovery via another machine - if the
server and the backup machine with the tape drive died, we would have
great difficulty getting the data back. For many years now, I've used
rsync to two separate machines in two locations, with hardlinked copies
to give snapshots without taking more space than necessary. Of course,
the type of backup strategy you need depends entirely on the situation
and the quantity of data. |
|
|
| Back to top |
|
|
|
| Unruh... |
Posted: Wed Oct 28, 2009 10:11 am |
|
|
|
Guest
|
terryc <newsninespam-spam at (no spam) woa.com.au> writes:
Quote: On Wed, 28 Oct 2009 14:57:55 +1100, Grant wrote:
Quote: *sync isn't a abckup system. It is just a copy system.
And a backup is not a copy?
A backup system has multiple copies, with historical content.
man rsync
Look for --link-dest
And if you want it slightly more automated, use rsnapshot, which moves the old
directories for you. |
|
|
| Back to top |
|
|
|
| Unruh... |
Posted: Wed Oct 28, 2009 10:15 am |
|
|
|
Guest
|
David Brown <david at (no spam) westcontrol.removethisbit.com> writes:
Quote: terryc wrote:
On Tue, 27 Oct 2009 21:48:21 -0700, Keith Keller wrote:
On 2009-10-28, terryc <newsninespam-spam at (no spam) woa.com.au> wrote:
On Wed, 28 Oct 2009 14:57:55 +1100, Grant wrote:
*sync isn't a abckup system. It is just a copy system.
And a backup is not a copy?
A backup system has multiple copies, with historical content.
How do you come to that definition? A backup system has as many copies
as desired/needed by business practice. If one copy without history is
sufficient, then that's a good backup.
The basic reason is that problems can happen with "the copy". No matter
what your system, one copy is no copy at certain times.
For an example, consider what happened to a very large software company
recently when trying to do a hardware upgrade of the storage system.
There was only one copy of the data (an old copy at that). Before doing
the hardware upgrade, they wanted to make a new copy - but there was not
enough space on the backup system. So they deleted their only copy, and
started making a new copy. Since it was going to take days to make the
new copy, and they couldn't be bothered waiting, they cancelled the copy
and did the hardware upgrade anyway. When that failed, everything was gone.
As you say, having a single copy is not a backup system by itself.
Sorry, but your example does not show that at all. No backup system is proof
against total human stupidity.
Quote: And even multiple copies on multiple devices at multiple locations is
not a backup system - you need a tried and tested recovery procedure to
make it a backup system.
And rsync is a recovery system as well. That is the huge advantage of rsync. What
is saved is the same as what is needed. No compression, no weird format, not
concatenation that can be unreadable if a single byte changes.
Quote: As others have pointed out, it is the system, not the tools (tar, cpio,
rsync, etc).
However, the tools form the central part of the backup system, and
determine the main features you have available. Thus it does make sense
to talk about an rsync backup system.
As you say, it is about what he really needs.
My 2c is that depending on amount of data and frequency of changing, he
could probably just dump his databases and just burn a DVD each night.
$AUS50 for a dvd burner and AUS50c/DVD. Even cheap DVD should last a few
years before they are toasters. |
|
|
| Back to top |
|
|
|
| Günther Schwarz... |
Posted: Wed Oct 28, 2009 1:12 pm |
|
|
|
Guest
|
terryc wrote:
Quote: On Tue, 27 Oct 2009 21:48:21 -0700, Keith Keller wrote:
On 2009-10-28, terryc <newsninespam-spam at (no spam) woa.com.au> wrote:
On Wed, 28 Oct 2009 14:57:55 +1100, Grant wrote:
*sync isn't a abckup system. It is just a copy system.
And a backup is not a copy?
A backup system has multiple copies, with historical content.
How do you come to that definition? A backup system has as many copies
as desired/needed by business practice. If one copy without history is
sufficient, then that's a good backup.
The basic reason is that problems can happen with "the copy". No matter
what your system, one copy is no copy at certain times.
As others have pointed out, it is the system, not the tools (tar, cpio,
rsync, etc).
As you say, it is about what he really needs.
Well said. The OP did not show up again. So it is not easy to guess what
he needs. One aspect that I miss in the discussion is networking. This is
a networking group, and the OP is a system at school which will likely be
a small network.
If more than one machine has to be included in a backup scheme a simple
script based system running on every single host quickly becomes too much
demanding in administration. In such a situation the big packages like
bacula, amanda or TVM show their strength in maintaining a single
database and backup documentation for all hosts. They are also able to do
load balancing and thus distributing costly full backups over the backup
cycle.
Günther |
|
|
| Back to top |
|
|
|
| David Brown... |
Posted: Thu Oct 29, 2009 3:46 am |
|
|
|
Guest
|
Unruh wrote:
Quote: David Brown <david at (no spam) westcontrol.removethisbit.com> writes:
terryc wrote:
On Tue, 27 Oct 2009 21:48:21 -0700, Keith Keller wrote:
On 2009-10-28, terryc <newsninespam-spam at (no spam) woa.com.au> wrote:
On Wed, 28 Oct 2009 14:57:55 +1100, Grant wrote:
*sync isn't a abckup system. It is just a copy system.
And a backup is not a copy?
A backup system has multiple copies, with historical content.
How do you come to that definition? A backup system has as many copies
as desired/needed by business practice. If one copy without history is
sufficient, then that's a good backup.
The basic reason is that problems can happen with "the copy". No matter
what your system, one copy is no copy at certain times.
For an example, consider what happened to a very large software company
recently when trying to do a hardware upgrade of the storage system.
There was only one copy of the data (an old copy at that). Before doing
the hardware upgrade, they wanted to make a new copy - but there was not
enough space on the backup system. So they deleted their only copy, and
started making a new copy. Since it was going to take days to make the
new copy, and they couldn't be bothered waiting, they cancelled the copy
and did the hardware upgrade anyway. When that failed, everything was gone.
As you say, having a single copy is not a backup system by itself.
Sorry, but your example does not show that at all. No backup system is proof
against total human stupidity.
The point is, before they started this upgrade process, they did have a
single copy. But they did not have a backup system.
But as you say, no backup system is proof against this level of
stupidity and irresponsibility.
Quote:
And even multiple copies on multiple devices at multiple locations is
not a backup system - you need a tried and tested recovery procedure to
make it a backup system.
And rsync is a recovery system as well. That is the huge advantage of rsync. What
is saved is the same as what is needed. No compression, no weird format, not
concatenation that can be unreadable if a single byte changes.
Absolutely agreed. |
|
|
| Back to top |
|
|
|
| David Brown... |
Posted: Thu Oct 29, 2009 3:51 am |
|
|
|
Guest
|
Günther Schwarz wrote:
Quote: terryc wrote:
On Tue, 27 Oct 2009 21:48:21 -0700, Keith Keller wrote:
On 2009-10-28, terryc <newsninespam-spam at (no spam) woa.com.au> wrote:
On Wed, 28 Oct 2009 14:57:55 +1100, Grant wrote:
*sync isn't a abckup system. It is just a copy system.
And a backup is not a copy?
A backup system has multiple copies, with historical content.
How do you come to that definition? A backup system has as many copies
as desired/needed by business practice. If one copy without history is
sufficient, then that's a good backup.
The basic reason is that problems can happen with "the copy". No matter
what your system, one copy is no copy at certain times.
As others have pointed out, it is the system, not the tools (tar, cpio,
rsync, etc).
As you say, it is about what he really needs.
Well said. The OP did not show up again. So it is not easy to guess what
he needs. One aspect that I miss in the discussion is networking. This is
a networking group, and the OP is a system at school which will likely be
a small network.
If more than one machine has to be included in a backup scheme a simple
script based system running on every single host quickly becomes too much
demanding in administration. In such a situation the big packages like
bacula, amanda or TVM show their strength in maintaining a single
database and backup documentation for all hosts. They are also able to do
load balancing and thus distributing costly full backups over the backup
cycle.
Günther
We can talk about rsync and networks if you like! rsync is very
network-friendly, as it only copies over the differences between
directories when doing a synchronisation (assuming you have appropriate
flags set). For large files, it can even copy only the changed parts of
the file. And all the network transfers can be compressed. This all
makes it very bandwidth friendly, and is very useful for doing offsite
backups.
For backup of multiple machines, it's best to install an rsync server on
the servers, and use an rsync client on the backup machine. This gives
you a single place to organise most of the backup system. |
|
|
| Back to top |
|
|
|
|
|
All times are GMT - 5 Hours
The time now is Sun Dec 06, 2009 11:00 pm
|
|