Lessons

Lesson Info

Extended Workflow: Back Up Images

I wouldn't say we've glossed over stuff. We've gone pretty deep on most of the key things here, but we've going at a fast clip. And there is a couple things here at the end of the day. We are gonna talk about today. Starting off with session, we are gonna talk about backing up your images. This is a key thing. This is a huge deal 'cause if you don't back up your images in this digit world, you are gonna lose them. And I've got 25, I've got 250,000 slide images that have been in my closet for five years. And I've digitize a bunch of those but it's a little easier to keep track of actual physical images and slides, and not have to worry about so much that you're gonna to lose something, unless your house burned down, which is horrible option to think about. With digital, we can now actually back stuff up as many times we want and it's an exact copy of the original. And how you back up will probably dictated if you lose images or if you don't. Of course, we've already talk about some of t...

hese things like how you name your folders, how you name your file names to make sure you're not actually accidentally overriding stuff. If you accidentally drag an image from somewhere into a folder you didn't mean to. But in reality, this is just risk management. This is just like, how importation it is to you that your images don't get lost are destroyed or corrupted. For the average person out there, images are still important. It don't matter if they are prints in a shoe box or if they are pictures of your family. If you ask people what's the first thing they gonna grab when they run out of their burning house. It's probably the pictures of the family or their picture or whatever it happens to be in terms of images or video. And for lot of us now that could just be on your phone and in the cloud, so may not be a physical hard drive that we pick up, but it depends on what you're doing. For me as a pro, if I lose my images, I lose my career. So that's a pretty huge risk to take, if I'm not backing up, the way I should be backing them up. But here we're talking about all kinds of things, solar flares you probably don't think about solar flares. A huge burst of electromagnetic fields. These hard drives are magnetic so that's a risk. That's not something we really think about. EMPs, an EMP is probably some bomb or something horrific that we don't even really wanna go there thinking about that. But that's a risk in terms of long term storage of images. Tornadoes, I live in the South west. We don't really have tornadoes in New Mexico but in Texas right next to me or in the Midwest in United States. I've seen a lot of tornadoes growing up. Hackers, that's one you may not thought about. Like if your hard drives with all of your images on it are on a drive that is connected to the internet, people could break in and start deleting your stuff. Or holding hostage like it's being happening to Sony, and a the whole bunch huge corporations around the world these days. My brother actually, well I shouldn't say, 'cause he got some pretty high security clearances, but I know some hackers who are incredibly skilled, and could break into my system like that. So hard drive crash is the big one. All hard drive is gonna fail. It's just a matter of when. So keep that in mind so as we we're talking this. Here we go all, hard drive going fail and that's across the board. It doesn't matter what brand we're talking about. They have different rates failure depending on just over time that people have measured. I'm not gonna trash anyone hard drive company. There's only three hard drive companies out really. But we'll get that here in a bit. So the basics for baking up. At a minimum, and I consider this dicey Have your images in two separate places, meaning they're on two different devices. So either on your laptop or computer and on a second hard drive. In my world that's not even considered back up. That's just considered due diligence for the start of a back up plan. If you really care about it, minimum of three three places. Bear minimum three places, and one of those places should not be, if you've got on your laptop, on an external hard drive, and on a second hard drive or on a DVD or whatever capacity of images you have. That third back up should live somewhere else. It shouldn't be at your house. Maybe it's at your office. Maybe it's at the safety deposit box of your bank. Maybe it's at your parent's house or whatever it is. And maybe even in a different state would be great or it's up on the cloud, which who knows where that's at depending on which service you have. But I'd say three back ups or three copies of whatever you have full copies of everything in terms of your photos at least is the bare minimum standard for backing up your images. So let's keep going here. There is in the computing world something called the 3-2-1 Rule, which is basically two copies on different media, which is a little more difficult to do these days. And one copy in a remote site. So we've already talked about that to some degree. In the old days, I used to have one of my copies of my images be on DVDs. But because files have gotten so big, and DVDs didn't get any bigger, that's pretty cumbersome. These days most people just have them on hard drives. You could put it on tape if you wanted to. I mean really big agencies and scientific institutions still have magnetic tape. And they put stuff on a magnetic tape, and that's actually more hierarchical probably than a hard drive. But for most of us individuals, that's pretty cumbersome to think about and very difficult to actual get data off the tape. 'cause it takes a lot more time. The other thing here is it depends on how much data or the size of your image collection is. And one thing I will add to this is I would highly suggest that you have one folder on whatever hard drive you buy, only images go on that hard drive. And have it in one folder and then all your sub-folders in there with all your different shoots or however you're gonna organize it. But don't put anything else on that drive that's holding your images 'cause that's just gum come up the works. And you're gonna slowly add to that hard drive, and fill it up at some point, and have to replace the hard drive, but keep it simple. Your laptop has no images on it, but that's what your looking at your images on. This is your first working drive here, and then your back up. This is a very basic back up strategy. And if you have up to around 10 terabyte of images. These days I think Seagate, maybe Hitachi too are now making 12 terabyte drives. There's one company that has even a 14 terabyte drive. Individual, single drive these days is not, I don't know how tested out that 14 terabyte drive is but if you have up to 10 terabytes, you can buy a single drive, and have your entire collection of images on one hard drive, and then backed up to another hard drive. It gets a little sticky. Well here's an even better back up charge, so we have it on three places. And then maybe I'd take this third hard drive and put it in my safety storage, or safety deposit box at my bank which is what I do. And I'll show you my strategy here in a bit, but we'll get there. And so you have three copies of it at any given time. The trick with this though is that this is at the bank, or wherever it's at somewhere different. You've gotta back that up every once in awhile because if you just have these two copies at your house or at work or wherever you're at, and your house burns down, you might lose a shoot in between the time that you backed those up. So you might even have two of these over here, and you just rotate them every week, or every two weeks or how ever often you do that. So I would say three copies. If it's important, three copies minimum. And I'll share with you some horror stories. I think pretty much every professional out there has a horror of losing an entire shoot. I know of some, some of my friends have lost an entire three week shoot. For me I had to, let's see hard drives or the exact same type of hard drives. They were sitting right on top of each other. And both of those hard drives went down within 24 hours of each other. And they didn't have my images on them, but they had all of my business stuff for my entire career on them. And I nearly lost all of my business information for my entire career because those two hard drives went down. I also had two other hard drives go down a few months later where I lost an entire shoot. A mountain biking shoot in Surbition, Colorado that I did for a bike magazine. And when I sent the images to Bike Magazine, they had me, this is back in the day before we started uploading images because of fast internet. They would have me send in a DVD because they wanted my worked up TIP files, but the also wanted the raw image file. And so they took, 120 of my best images for the shoot out of 2000, 3000 images that I'd shot over two or three days for them, and that DVD. So when those hard drives failed. It must have been eight years later. I called up David Reddick of Bike Magazine, who's the photo editor, and I'm like Dave is there any chance you still have that DVD I sent like a million years ago? Thinking there's no way he's kept this thing. And he goes through his file cabinet and he says, "Oh yeah, I got it right here." Like dude, can you send that back to me because those are the only images I'm ever gonna see again from that shoot 'cause I lost all the other ones. So as you lose a few images, you're gonna set up really set up really quick, and start figuring this stuff out, which is what happened to me, which is what happened to every pro photographer in this digital age. So if you've got more than 10 terabytes, then this is just a random. It's not a random number but it could be 12 terabytes or 14 terabytes or once you get above 10 terabytes of information. It becomes a little harder to store that on a single drive. So for me, this is what I have. I have these RAID Arrays which are basically in a closure. It's a metal box with four hard drives in it that shows up on my hard drive as a single hard drive. The images are being stored across those four hard drives, and we'll talk more about this in a bit. What those exactly are. And these are 24 terabyte enclosure so I can hold up to, depending on how I have it set up either 18 or 24 terabytes of information on there. When they make these now, you can buy RAID enclosures. Basically unlimited storage. This is what a cloud data center is. They're not boxes like this, they're giant rack mails systems that literally have thousands and thousands of hard drives in them. And this is what the internet is. If you've never understood what the internet is. It's giant hard drive arrays all over the world storing your website information, so that it's accessible. And how fast your website is accessible is how fast those cables connecting those hard drive can shoot that information over to your computer, about whatever is on that website. This is a microcosm of data centers basically on your desk. And it does get tricky for that third back up because if you've got 24 terabytes information, but the biggest hard drives you can find are 12 terabytes drives. You might have multiple drives that you're putting in your safety deposit box to back up whatever archive you have. For those people out there shooting video. This becomes a massive nightmare. Like we shoot with red digital single cameras in 6K or 8K for a lot of our jobs. And you're shooting one to two terabytes a day. Imagine how fast you start. You might as well start buying stock in hard drive companies. if you buy a red digital cinema camera because you're gonna be spending a truck load of money on storage. And in your office, you might even have rack storage like they have here are Creative Live. They have massive amounts of storage to store all this video they're producing and to put it out live to the world, so you can access it. Storage becomes this nightmare thing that you may not have thought about especially if you're producing a huge number of images. And especially if you need to access those images at any given date in the future like most professionals photographers need to. So here it still is 3-2-1 Rule. My computer, the first set of images and the first copy of images, second copy of images, The ones go in the safety deposit box to get out of my house or going to my parents, and then maybe even have the cloud over here. And you're uploading some images to the cloud. We'll discussed that as well. Here is a confusing graph, and don't wanna overwhelm people. Remember what we've gone through. The 3-2-1 line. There's three different places. Take one out of your house, you're pretty safe. And then the other caveat to that is replace all those hard drive every three to five years minimum. Because hard drive feels. There was a test. I remember a gentleman. I can't remember his name spoke. He is a computer guru in Albuquerque that does stuff with the U.S. government, and showed us some graphs that the U.S. government is testing on hard drives in all media in terms of archival storage. 'Cause obviously the government has lots of information they wanna store for long periods of time. And they filled up hard drives to 80% of the capacity. And they filled up DVDs, they filled up every known media. Put it ina safe for six months, and pull it out. It wasn't used for six months. A good chunk of those hard drives were not even readable. So that's pretty scary to think about. And if those hard drives are being used every single day. You have the option of things going bad 'cause typically how hard drive goes bad is a spinning disk drive. A hard disk drive as oppose to a solid state drive, which is like the flash cards you put in your camera is the heat builds up, and the disc warped, and then it cannot be read by the arm-iture that goes over the disc. So that's 99% of why hard drives go bad. A side note here. A little tip if you do have a hard drive going bad and it's a hard disk drive, put it in your freezer for 30 minutes and see if you can read it. That's what I did with my LSC drive, when I nearly lost my business stuff. I was online and the forums and it was oh my gosh, this is insane. I'm gonna lose 10 years worth of my business. I'm starting from scratch here on all this information I had about clients. This person told me put it in your fridge or your freezer for 30 minutes, pull it out and reading it. And somehow it cooled down the drive, and the drive became flat. I don't know exactly what happened, but I could read that drive for the next hour, and I had literally just enough time to copy everything off a it before it died again. And I stuff it in the freezer again at that point, it never came back. Little tip, I've actually had people email me about this, and I've told them the freezer trick, and it's worked for just about everybody who's emailed me. One time, not like multiple times. Apparently it only works once, but if it hadn't been for that whole holy mackerel. So that just makes me sweat just sitting here thinking about that 'cause that's pretty dicey. So I became totally just like everything else you've seen in this digital workflow. I learned everything I possibility could about backing up my images, about hard drives, and how they work. I had a clue before but I definitely know more now. And this is basically my back up work flow to show you what I do. Camera is over, these are hard drives I take into the field. they're just little, portable hard drive from G-Tech. Backing up, that thing I said about hard drive and heat dissipation. Whatever hard drive you're gonna make, or you'll buy excuse me. Make sure it's got some form of venting or cooling built into the enclosure if it does not, no not buy that hard drive. The metal cases let the heat dissipate faster than a plastic case. Keep that in mind. If there's a little fan built into it. Most of these portable drives do not have fans built into them but G-Tech is very smart, and that they have created holes and there's a cooling and venting system in there. They even have the bottom of them are sometime crinkled so that it cools faster. I do have some of these now that I use are not hard disk drives anymore, they're usually solid state drives just 'cause the prices come down. That's what I take on the field. Those are usually, I think I've got several one terabytes left over now. Most of my are two or four terabytes hard drives. My solid state drives are one terabyte drives 'cause you're starting out at two terabytes. All the state drives get pricey really fast. And it's almost a matter of how big the data you're collecting in the field is, and how fast you wanna move that over to a hard drive. 'Cause the hard drive as we talked about earlier maybe 180-200 megabits per second is how fast that drive can reiterate data. If you've got terabytes of data you're trying to download that could take hours to get it under just one hard drive, and then another couple hours to get it onto that second back up drive, before you have to clear the cards to shoot the next day. That's why I've moved the solid state drives for these portable drives because I don't wanna sit there for an hour to download those 9000 images and have to sit there for another hour while it does it in the second time. And the read and write times can be drastically different. If it takes an hour to download that huge file to a spinning disk drive. It might take seven minutes on a solid state drive. Because it's 600 megabits per second read/write speed on a solid state drive is massively faster than that 120-140 depending on how full your drive is. It's on the spinning disk drive. So I know we're getting super geeky and techie here, but this stuff all starts to add up, and you're gonna figure out sooner or later when you start downloading and moving images around. So whatever drives they have on the field, I have this G-Tech Gdoc, which is basically a dock to plug those drives in. That's not something you actually have to have, but it makes it convenient for me 'cause often I'll have three or four different drives with data on them especially if we're shooting video. Regardless, you just plug whatever drives you have in the field or out on your shoot. This is a Mac Pro, which I used to use in the computer, and then those drives go up to my first live work back up here. On this live work back up is my working drives where images sit while they're being worked up. And the reason I have this as opposed to just going straight to the drive where they're gonna live forever is because these images are being put on there, and they're being worked up. They're been saved and re-saved, and when I put the finished final master images over here on my archive drive. I wanna do that once, and then probably never touch them again. So that it's a very clean drive, and how it's written to that drive. That's not to say that I won't go in at some point, and change some stuff, and make sure it's backed up everywhere else, but that's rare. The other things which we'll talk about rate zero, and rate five here in a second because my archives is 20 terabytes, and that's just the stills. The video is another 20 terabytes, I have more than 72 terabytes of drives just sitting on my desk. So that's not even all the drives I have with all of my stuff on it in the safety deposit box and counting the back ups off site. That's just because you have as a professional, a much bigger catalog of images, much bigger data sets. That's minuscule compared to to what Creative Live has. We're talking petabytes of information I sure. So the other thing, my computer is backed every night onto another hard drive, so it's not just my images. Not this laptop but my main working laptop at home is backed up, there's a duplicate of this entire operating system that is bootable on a second drive, and then there's also a copy of that at my safety deposit box. So that if my computer, and I have two of these exact same laptops. I think the hard drives inside the laptops are a little different size, but they're pretty much identical. If one of my computer just goes down, and I'm in the office and a client needs something. I can take my other computer, plug in the hard drive that was backed up the night before boot up off the hard drive, and I'm back to where I was within five minutes. Whereas if you don't have a separate computer, and you're not backing up your main working computer. It could be three days before you get a computer and get all the software set up, and everything else dialed in to get something out. And for me that's just not something, clients aren't gonna wait three days to get the image they needed yesterday. So it's something I've done. Anyway, this live work drive is basically a drive that I work everything upon, and that gets backed up to another drive every night as well. I use SuperDuper, I think I said it earlier. It's a $25 thing for Mac Computers that you can download on the internet. Just look up SuperDuper.com. I think SuperDuper is the name of the software. It's a different software company that actually creates it not SuperDuper. And there's plenty of these out there like ChronoSync X. There's a bunch of these different back up software options that you can buy to back up your stuff on a nightly basis or however often you want to back it up. That's my working drive. It has my layer on catalogs on there. It has all kinds of other stuff besides just the images. All my business stuff is on there 'cause it's a 12 terabyte drive. And it's in a rate zero configuration which we're gonna talk about in the next session. So I don't wanna get too crazy about the usual rates configurations. Honestly for most photographers, if you've only got four or five terabytes of drives, stick to single drives. Buy a brand of hard drives that you like. Make sure there's a fan or some cooling option in those hard drives that you're buying. They're probably external drives. Buy three of them, and make sure everything's simpatico and the same on all three drives, and you're probably good to go. If you're out in the field. I have a photographer friend who also backs up to the cloud, just as insurance. Is that something that you've done? No, because like those Bi-hiee shoot at Jaws. I shot 9000 images on a 46 megapixel camera. It'd take a week to back it up on the cloud, and typically your internet is not that great in most hotels unlike here in Seattle where it's pretty amazing. The cloud is something I do at the very end, and only with my best images. The other issue with the cloud that I didn't talk about is getting your images off of there if something goes wrong. People how offer cloud services are generally pretty dialed in on this stuff. So they're not gonna lose your images, and they have all these giant RAID Arraies, so that they can pull out a drive if it goes bad. Put a new one in and it rebuilds everything. And nothings is lost, which we'll talk about here in a second. But it's accessing your stuff. There was a company Digital Railroad years that was made for photographers as a cloud service back up to license their images, and also to back them up. And they went out of business, and they sent an email to all their customers. Thousands of photographers said you've got 24 hours to get your images off of our servers, before we shut them down. There's a thousand photographers trying to get terabyte images of this. Not happening, so you can't rely on cloud to be there consistently. You have no idea how good it is or where it is. Your cloud back up could be somewhere where there's giant hurricanes or Fukushima or where ever. I'm just throwing out random names here. I don't know if there's anything there, and you could lose everything on the cloud. So that's like a extra bonus back up, not something I would really rely on. Tony. I've got seven and a half terabyte on a 10 terabyte drive, you mentioned 80%, so I shouldn't really, I'm getting close, ain't it? Well typically most tech guys don't recommend that you max out hard drives. You probably wanna leave 5%-10% at a very minimum, empty on a hard drive. I've got a few hard drives that are pretty maxed out, and I keep telling myself I need to take some stuff off of there and put it on fresh hard drives. But 80% is a good number, not to go too much past that. Just like your hard drive here. If you fill this sucker up, it's not gonna work so effectively. If you fill up an external hard drive, you've got so much stuff packed under there, there's no scrap disk left on that drive to allow that hard drive to do anything else. 'Cause when you buy a hard drive, it's not just the drive itself. There's some hardware in there to actually help you use that hard drive. So filling it up, another way hard drives go bad. Is it loses track of where stuff is on the hard drive because of corruption within the components of the hard drive. You don't wanna push the envelope too hard on the hard drive. That's my advice. There's probably people out there that know, well not probably. There's certainly people out there that know way more about this. Chime in if you're out there and have a better understanding of how far you should fill up a hard drive. That's what I've been told by people that I trust. Yeah, Michael. Yeah. A couple of questions from the internet. I don't know if you answered this so I apologize. Curious about what makes a hard drive go bad in a safety deposit box? What could cause problems there? Any guesses? I don't know. The funny thing is if you look it up, hard drive reliability statistics online. It's different for each of the three hard drive companies, but on average it's between 2%-18% failure rates, and 18% is really high. That's an anomaly. Usually it's between 2% and 8%-10% failure rate. And typically hard drive will fail within the first six weeks to six months, if there's something wrong with the drive. Otherwise they'll usually last three to five years, if they're cared for. In a safety deposit box, it could be any number of this or if it's not something you can regulate. It could be EMP, it could be who knows what's going. It's necessarily, the solar flare went off, and hard drive at your bank down the street. And didn't kill yours. There could be who knows somebody is walking in there with giant magnet, and putting that in their safety deposit box. That could certainly affect your hard drive, so there any number of things that could happen. It's in a steel box which is technically fair day cage. Meaning that it blocks transmission of electronic symbols, so hopefully that's good. There are these cases, let's see if I can remember the names of the cases. They have some very strange names. I'll have to think about it here. They're in plastic boxes that are lined with foil, so that they enclosed the hard drive. And there are little fair day cage for each individual drive. And that's what I use to put in my safety deposit box for individual 'cause the drives I use are not in closures like this. They're just raw hard drives 'cause it's a lot cheaper to buy just this drive than to buy this whole enclosure with the drive. And it's also a lot smaller, so that I could fit more of them in my safety deposit box. I'll have to look up the name of these hard drives case. I didn't think about putting that in the show. But that's a good question. It could be any number of things. And you mentioned three to five years, so that's about how often you like to replace your drives. That's how I do it. I know the gurus said every 18 months to two years. I'm like holy macaroni I'm gonna go broke on hard drive replacement here. And if you have 20 terabytes, you need to back up in three or four different places. That is an insane amount of time to copy stuff over to new drives. What I typically do is I replace my main hard drives, and then set those hard drives down the chain to become the back ups in the safety deposit box. And they are maybe two to three years old, and they'll go for another couple of years and then I'll eventually just keep putting fresh drives in, and keep moving those down. And that's how I do it, which maybe not be the best possible way. But it's an economical thing as well. Luckily these days, a three terabyte hard drive we can get for less than $100. So hard drives have gotten very expensive. And then one thing Rosewell. Rosewell? Rosewell is the case. Oh, I've never heard of that one. The one that people are mentioning, and then also you said as long as take care of them. The drives like two to five years if you take care of them, What does take care of them mean? As you don't work with your hard drives in office that's at 102 degrees. Cool. You don't necessarily need air conditioning depending on where you live. Certainly here in Seattle, it may not get that hot in the summer. I don't have air conditioning where I live in New Mexico because we're at 7000 feet altitude. So around 2300 meters and it rarely gets above 90 degrees. Every once in a while in the summer it might get above that but we also have Adobe house that keep it really cool on the inside. But if you're in Phoenix, Arizona where it's 117 outside in the summer. Like half the summers these days. If you don't have air conditioning or some kind of cooling, that drive is gonna overheat really fast. There's another thing I haven't talked about here which maybe another question somebody asks. Enterprise drives versus just your standard, normal drive. There's what's call an enterprise class hard drive, which looks identical to basically what you'd normally buy at Best Buy or any electronic store, You probably can't find enterprise drives at your local store. Enterprise drives are designed to run 24/7 in data centers. And they are built to a higher tolerance. So that their fault tolerance is usually lower, but the price is usually double or two and half to three times what a normal drive is made to do. In a normal drive, this enterprise class drive, I think technically is made to run eight hours a day, and then be turned off so that the discs are not spinning. We're not talking about SSDS here right now. If you just work eight hours a day at home, and you gotta turn your hard drives off, normal drives are fine. If your data is super important to you, and you have the financial resources to buy the enterprise class drives, then those are gonna be more reliable. If you buy G-Tech drives, most of those are enterprise class drives. I can't say enterprise very well, and G-Tech is owned by Hitachi, who is one of the big three drive manufacturers in the world. So you know those are all Hitachi Enterprise class drives, but that's why G-Tech cost more. If your buying hard drives just of cost alone, you may not realize the difference in the specs here. You'd have to look under the hood of the specs, pretty seriously to find that out. Were there a few more questions? These are good questions. I think we're good to move. Alright, Rosewell? Is that-- Yeah. I have to look that up. Mine started with a Z is like Zeffer or Z. It's a really strange name that I order online. Maybe somebody knows the name of that. It's like Z-E-E and then three other letters. I have to look that up myself. But there's a variety of cases made for storing hard drives in long term storage. And these hard drives in my safety deposit box. I definitely pull those out once every month or two, and plug them in to my stand alone hard drive reader thing. Which basically is just a vertical slot that you stick the hard drive in. I get it from Other World Computing and spin them up to make sure everything is going well. 'Cause I don't want them to not spin up because there's grease and there oils in there. If those dry out since I live in a very dry environment in New Mexico. The hard drive may or may not work as well as it could. That's another reason hard drives could go bad is if they're in there for years, all the lubricants in the hard drive might get really crusty, and one might hit the disk, or get stuck in between the disk. The other thing you may not realize is the platters. Final records for these really high capacity hard drives is not just one disk spinning. It four to six to seven or eight disk now. So there might be two terabytes of that 12 drives on one disk, and then there is six those two terabyte disk inside that enclosure. So think about the mechanics of how that works. And you're talking about trillions and trillions of zeros and ones that that thing is spinning at crazy, high speed to read these magnetic codes. What you don't wanna do is to stick a giant magnet on any one of your hard drives. I've got a magnet on the back of my phone to mount it on whatever surfaces when I'm driving in my car or this that or whatever. I'm just careful not to get that. It's a really powerful magnet too close to my hard drives because that could mess around with the data on that hard drive because everything is stored magnetically.