The Data Migration Thread

Yeah, but I have a thousand CDs that are already rotting! What else can I realistically do if I still want to hear the music in 20 years? Like you said, I'm on the hamster wheel.

As I reconsider that, perhaps the answer is "tape them on cassettes". I haven't mentioned it yet on this thread, but I am keeping all of my tapes just to see if they outlast the CDRs I made from them.

I never got back to your original question about what media the "future" will have, but also on the sandbox thread there seemed to be some consensus about labels only releasing lossless downloads (for those who just want the music) and vinyl (for those who want the object).

Revive. So this year I've been dumping god knows how many CDs onto the new hard drive and ditching them. A refreshing feeling, really.

redundancy?

Department of

Why the fuck did I sell my copy of TLS: The Fifth Mission (Return To The Flightpath Estate)

also wound up repurchasing YMO's Multiplies for the THIRD TIME as well as tracking down a copy of boom boom satellites first album which I have no fucking clue how I managed to get rid of

but everything is now ripped in satisfactory form and backed up so I should be ok for the foreseeable future. knock knock.

in the process I wound up having to acquire yet another big case logic binder, a 500 GB external hd and an external optical drive since I blew out the one on my iMac. I believe I've related elsewhere the fateful joke that the internal reader's last hurrah was (and remains, since the fucking disc is stuck in there, perhaps permanently) cristian vogel's "All Music Has Come To An End"

how much money you getting for your stuff, ned??

I made a nice four figures selling my bullshit on amazon just a couple years ago or so

TOMBOT'S Amazon Marketplace Blowout (wherein you make jokes at the expense of my juvenile obsessions)

how much money you getting for your stuff, ned??

Some pretty reasonable hauls via Amoeba, actually. I'd say for this year alone it's been well into four figures. A LOT of randomness I've been shedding (keep in mind that there's still thousands of CDs left -- that'll change).

keep in mind that there's still thousands of CDs left

tower records r.i.p., amirite???

;-)

Actually that's a pretty small amount in comparison.

So, to those of you who have decided to migrate the majority of your physical collection over to the digital realm, how is that going?

Recently, I decided to go digital as much as I feel comfortable. I decide to begin a partial purge of my CD collection, selling things in waves until I'm only left with my favorite records, which I want to hold onto for archival purposes. Anything that isn't essential is getting sold, and I've started buying new records through iTunes and Amazon mp3. It seems more or less inevitable that most people will either be buying or streaming new music online in the near future. It's kind of already gotten to that point, as my good local indie store doesn't reliably have copies of all the new records I'm looking for.

For those really interested in this subject, Maura over at Idolator recently posted an entry, What Have You Done With Your CD Library? (If You’ve Done Anything With It?), which started an interesting discussion in the comments section.

*decided

I haven't migrated the majority over, but I am up to probably 500 (CD and vinyl, all ripped) so far, with 150+ CDs on the chopping block. I'm still trying to rip most of the stuff to flac as well as a straight CDR copy, but some of it I just don't care enough about. I would never consider replacing any sold disc with MP3s, gotta have lossless. I can live with whatever sonic ephemera comes with digital copying and manipulation, so far I haven't heard anything that gave me pause except for the aforementioned CDR disc rot, a single bad DVDR, and World Serpent problems.

As far as checking the condition of older stuff, I have mostly been going back and checking the things that were ripped under unusual and dubious conditions in the early 2000's (CD player straight into pro CD burner in real time, hence limited track marker ability). The levels mostly came out OK, only one so far was overmodulated (Ora's Aureum 2LP, bummer cause that was my fave). I have been using Peak to fix some of the original tracking problems and saving the results as new flacs.

Like Ned noted somewhere on here, I have not really missed a lot of what I sold. But I have also pulled out backups for a radio show or just to play and been glad I had them.

also I wish I had possessed this technology when I was selling tons of stuff at record shows in the mid-90's.

I sold every CD I didn't care about without creating a backup. Then, I imported everything that was really important to me to my computer in 192kbps, saving the CD as a physical backup. I had a rather modest CD collection, which I don't think ever exceeded 200 records at one time. So far, I've sold at least sixty records, I think, with more to go probably.

I'm not an audiophile, so I don't mind buying new records from iTunes or the Amazon mp3 store. 256kbps sounds good to these ears. And while the fact that mp3s are ephemeral gives me some pause, I appreciate the fact that they don't take up anything except for hard drive space. One of the big impetuses behind me selling so much back in the first place is to just have less physical stuff cluttering up my life.

Has anyone else been working on any data migration-related projects lately? Continuing any old projects? Anyone else getting rid of most of their physical CD collection?

Well you linked that Idolator post so I already weighed in there but I'll c/p what I wrote there here:

--

As any number of friends of mine know my CD library is a bit...large. (The term “Raggettstacks” was coined some time ago.) That said I’ve actually been reducing it bit by bit over the years, as I sold off CDs in waves just to clean the place up a bit — even so, however, we’re talking a huge amount. I had always had vague ideas about making some sort of continuing archive out of it, but wasn’t too sure what the best approach would be — while I work for a university library, I didn’t necessarily know if there would be interest in it here, especially since it is fairly idiosyncratic.

That all said, at the end of last year for a variety of reasons I decided it was time to bite down hard and make arrangements to pass most of it on, outside of a variety of discs kept for their rarity (especially CDRs and limited run efforts), for sentimental reasons or just because I could never see myself actually parting with them under any circumstance. It’s still a fair amount of discs but the amount of what I wanted to let go completely overwhelmed it.

After thinking about selling it all off in a heap, then chatting with a couple of friends who urged me to consider finding some way to keep it together, a good notion occurred — I still live near where I did college radio all these years, and am in touch with both current station staff and a couple of old veterans still there. So I decided to see if there was any interest on their part for a donation of discs — I’m pleased to say that there was. Since then I have been engaged in a thorough effort to back up each of my discs (and make backups of the backups) with an eye to completing the mass donation by the end of the year.

By giving the collection to an organization that is dedicated to music, as well as to maintaining a deep catalog, I figure that it’s the best of both worlds — doubtless there will be some duplication but there’s always room to fill some gaps, while any extras could conceivably be used for fundraising efforts or the like. Either way, by benefiting the station, both now and in the future, I like to think that I’ve made a positive contribution to an organization where I made and maintained friendships to this day, and which in no small part shaped who I am as a listener as much as a DJ and presenter. Lord knows I spent a large amount of time scrounging through the station archives turning up music I’d never heard of before — why not encourage this cycle to continue, and to help secure the role of such stations to continue to share out music to listeners?

Financially this option might not be everyone’s — especially now. But I’d encourage people who might have this option to hand to consider it. On top of which, frankly, I look forward, given all that I do have, to not having to move any of it ever again when I do my next inevitable house move, whenever that will be!

Ned, I actually read your post over at Idolator, and this is such an awesome project!

If you don't mind my asking, what percentage of your CD collection are you planning on holding onto?

I'd say around...20%? And that'll actually probably skew downward after I go through that remnant.

In 2009 I did a major amount of ripping, getting down to under 500 CDs and CD singles left to go - the absolute dregs of my collection, much of which I need to consider selling. I try to get rid of 100 every year to make space for the new acquisitions, thus keeping a steady amount that fits on my rack space. Having all this music on my PC has allowed me to review old stuff that hadn't been spun in a decade, sometimes finding gems, other times making an easy decision to sell. I've got a big barrier about hanging onto the physical item that I've been slowly chipping away at. I'll never get rid of EVERYTHING, but I feel like I could probably ditch that last 500 and never notice. My ultimate goal is to get a couple of those large drawer systems (http://www.can-am.ca/), get rid of as many jewel cases as possible and reclaim a ton of wall space.

I've already bought 5 individual digital tracks, mainly from compilations that I already had most of. I expect that trend to continue - there's already a couple of digital-only releases I have my eye on but I'm waiting to see if they get a physical release as I'd always prefer to make my own rips.

I also have been ridiculous about managing my tags. I added cover images to over 3000 files, mostly singles, and try to be consistent about my naming conventions, my folder designations (Singles/Radio Sessions/Demos/Live for non-lp material) and preserving original album tracklistings while siphoning off bonus tracks to aforementioned non-lp folders. I mine for duplicates - I can't stand that, and it mainly impacts the integrity of compilations but that's how I like it. Speaking of compilations, I change the artist tags to some variation on "Compilation" because I don't want to see all the individual artists on my iPod or Sansa. I standardize the volume of everything by adjusting the gain. I do some other things to verify the quality of the MP3s themselves. It's taken me about 8 years to get to this point and I'm very happy with my system and my collection.

got a 1.5 TB external today, a Seagate Expansion. Starting at the beginning of around 250 DVDRs and getting them onto the drive. I'll be interested to see how the older ones have held up.

I am now on DVDR #4.

note: this is all flac

No matter how much music I have on my hard drive, I still end up doing my primary listening on CDs

all music collecting is ultimately just a pain in the ass and hoarding digital files is just as pointless (if not more pointless) as hoarding CDs.

yeah, I know. this is more for the convenience of not having to go dig through 200 discs to find some file or show or album.

all music collecting is ultimately just a pain in the ass and hoarding digital files is just as pointless (if not more pointless) as hoarding CDs.

yeah but if you have a hard time coming around to the "get rid of anything I'm not listening to" position, it's easier to have a hard drive filled with useless shit you don't need than it is to get rid of closets & attics & basements full of actual CDs.

Revive - a year on anything digitally related is akin dog years...

I now have over 600GB backed-up to two drives + 200 CDRs. Oberservation:.

1. I've had two HD failures in the last year (not the Caviars) but I was able to recovery 99% of everything with recovery software (Stellar Phoenix). This first recovery kept my USB-tethered HD spinning for about 6-weeks. Use of a USB 3.0 or SATA connection may have helped (but who knows given the black magic involved in deep level HD scans) but I wouldn't know because I was unwilling to turn off the machine until the scan was completed.

2. Adhere to your category sections. I didn't want to go too genre-centric. Since I'm not an academic, I've found it useful to make my categories more general. I know use the following:

Alt Rock
Folk - Country - Blues
Electronic - Ambient
Hip Hop - Rap
Jazz
Rock - Pop
Metal - Noise - Avant
Soul - R&B - Funk
World

These get me on the right track pretty quickly when looking for anything in general or particular. I probably have a couple other sections for holiday or kids music, etc., but the main categories have really help keep me from getting bogged down in the minutia.

3. When I archive an artist with a sizable catalog - i don't need everything. I've assembled a 1 CDR "best of" for artists like Nilsson, the Stones, Springsteen, etc, where I've weeded out a bunch of individual tracks. I found it also to be good exercise for examining the full arc of an artist's career. I've yet to exceed the space of a CDR with a condensed discography and a hi-bit MP3s.

3. When I archive an artist with a sizable catalog - i don't need everything. I've assembled a 1 CDR "best of" for artists like Nilsson, the Stones, Springsteen, etc, where I've weeded out a bunch of individual tracks. I found it also to be good exercise for examining the full arc of an artist's career. I've yet to exceed the space of a CDR with a condensed discography and a hi-bit MP3s.

This is a great thing to do, for the reasons you outlined, as well as the fact that some artists lesser material simply isn't worth the effort to get into.

I don't understand backing up to CD-R or DVD-R.

they last longer than your average hard drive (if you store them under reasonably good conditions)... I haven't had a failure yet, but if I do I sure will be glad that I have a reasonably dependable and cheap form of backup.

really? a lot of cd-r's longevity rates are much shorter than the manufacturer's advertised length. I suppose you need to buy quality brands.

yeah, the only failures I've had (like 3 out of 1000) were with noname brands. having said that, I should note that I have yet to purchase a secondary external, so a lot of what I have is ONLY backed up on DVDR or CDR. one thing at a time, it took me forever to rip them all. a 2.5 TB external will provide another layer of protection, then I think I'm good.

My mom is an archivist and they have an acronym - LOCKSS (Lots Of Copies Keep Stuff Safe). I've tried my best.

you need an off-site backup. I would back your stuff up to two HDs at least (plus the primary one you use daily), and keep one at your mother's house or somewhere else, in case your house burns down (knock on wood).

lol I just put my hard drives in the fireproof/waterproof safebox when I was in Peru this last month!

and I definitely plan on giving the next external I buy to a friend.

I was pretty guffed to find out that some fireproof boxes aren't really 'fireproof' - that is, they're rated as being able to withstand X temperature for Y time, but if the fire exceeds either then there's no guarantee. ;_;

while this thread is up, if anyone has any advice about how to get FLAC ID3 tags to work on a Cowon player using a Mac it would be very appreciated. I just got a new player today and I can see the folder structures (i.e artist/album cause that's how I have them labelled) and the track title, but the tag fields are blank. I use xACT for FLAC coding but I've never been able to figure out an ID3 interface.

This is a great thing to do, for the reasons you outlined, as well as the fact that some artists lesser material simply isn't worth the effort to get into.

― Gerald McBoing-Boing, Friday, April 29, 2011 12:54 AM (3 hours ago)

I also try to include at least one track from even the poorest of releases as it maintains the visual integrity of the discog as well as putting some the "sour" times in context with the better material.

Also, i neglected to include the obvious "Classical - Opera" category; I doubt, however, that any of my beautiful Columbia Masterworks and vintage Blue Angel slabs of wax will ever be turned into "1's and 0's".

http://www.backblaze.com/ seems like the obvious answer here. surprised it hasn't been mentioned yet.

> FLAC coding but I've never been able to figure out an ID3 interface.

FLAC uses its own built-in tags, not ID3 tags as such. but anything that rips to flac should know that.

not sure about mac but on linux there's a thing called metaflac which will list and modify the flac metadata. command line tool though, so you'd probably hate it.

(actually, i hated it, until i wrote a wrapper for it based on vorbistagedit which presents tags as a file that opens in vim and rewrites the corrections when you quit)

don't recall having a problem with flac metadata on my old Cowon M5 but that was some years ago. my current iRiver LPlayer chokes on ogg tags which aren't ARTIST=whatever - the ARTIST bit needs to be uppercase...

I back up to backblaze, but i've only got 75 gigs of music. 600 gigs would take a bit of uploading.

Check out Tag: http://sbooth.org/Tag/

CD-Rs may have lower failure rates, but you could just buy an additional hard drive. A 1 TB drive is like $80, that's 1,400 CD-Rs or so. Which I think would cost way more. Worried about both drives failing? Buy a third. I have had much more of a problem with scratched CD-Rs not able to be read.

Online back-ups are impractical with large stax. I'll stick too HD -- wish that Tape back-ups were still around at reasonable prices (why the hell does the hardware for LTO-5 cost $1K and up ?!?).

The cloud will save all? Maybe for the pedestrian listener, which means the products they make won't fit the high-volume customer.

I have yet to have a burnt CDR flake out. I keep them all in individual paper sleeves and out of direct light. Some are 8 years old and have been played dozens and dozens of times. Also, I only use Taiyo Yuden media which in my experience, has been nothing less than bullet-proof.

Got two backup disc drives today. Now I'm copying 300 gigs of vinyl/tape rips (in flac) to the first new one. 1.5 TB of CDRs will come next. Then I'm gonna take the backups over to a friend's house and leave them there. Finally, some serious redundancy - my old LaCie drive is almost dead.

Kind of a bummer that these new 3 TB drives only work with Leopard, I don't wanna upgrade (and don't even know if I can on an odler G4).

skip, thanks SO MUCH for that Tag recommendation, that program does everything I need it to. My Cowon portable recognizes any JPG or GIF as cover art, so I don't need to tag that art but just throw it into the folder.

Since I started this thread flac hardware support has gotten much better, I wish my Cowon understood shuffle play without repeating tracks but other than that I am totally happy with it.