ORIGINAL: Foxy
I dare say I am one of the more experienced IT professionals on this site, I have 24 years actual on the job experience, and currently consult on applications deployment strategies to one of the biggest companies in the world, but in the past I've worked on Bank data migrations as well as many much larger application migrations. You could even say that this kind of problem that RCU is facing is in fact my specialty.Having said that, I'm not going to say that the guys at IB (and yes I know a lot of the backoffice story around here too) have always displayed total competence and good procedures, HOWEVER... I have to admit that the enormity of this type of migration is of galactic proportions. Lest we forget, a few billion bank records does not even equate to the byte count of just the posts here on rcu, let alone the images, which would equate to TRILLIONS of bank records. Then there's the nature of the data, posts written in the different forum 'languages' - bbcode, html and the wysiwyg interface, all of which must be migrated to a single language, together with image conversion to conform to new site rules, changes in tags such as 'link' to 'url', finding a way to deal with corrupt posts, the number of which cannot be known, and finding a way for the migration process not to stop dead when it encounters data integrity issues, and finally migrating the actual configuration of the site for every user, every vendor, every moderator, every forum, etc.Then there's the thing about downtime... John, how exactly do you propose to move a bucket of water out from under one tap, place it under another, with both taps running, without losing any water. It is of course impossible, so at some point the tap must be turned off. An intermediary database could be used and then merged but it would have to have an identical schema to the main database and also somehow be compatible with both svhemas, it just isn't feasible for a deployment of this size. Much better a week of downtime than 2 weeks of extra work to prevent downtime. Yes, forums are simple, but this isn't just any forum...
Why I said in my 1st post another forum that had a mis-mash of formatting types said screw it, just deleted all the old posts, and accounts when they changed over last year. They didn't even have the picture issues to deal with, and said it would be too hard.
Hell remember when I had my buddy make me a DB that was similar to a forum(it was for actual data but the format the software used looked exactly like forum posts), and how it took him a week to scrape the data off about 700k entries for it, and half of which were corrupted in the translation.