Migrating Content is Like Moving House: How Did We End Up with all this Stuff? And where does it all go now?

Content is King, right? But alongside shiny new designs, new features and technical challenges, content barely gets a second glance. Much like when you move house, migrating your content seems easy enough until the deadline is looming and you’ve got no choice but to grab everything at random before it gets left behind.

Where do all the boxes go, and how do they get there? Do you even know what they all are? What order do they go back together in? And why do you still have that poster from the 90s? Moving house, and Migrating content, is all about planning, mapping and making a specific list and sticking to it. It’s also about knowing when to abandon everything and start afresh. Having migrated dozens of sites well, and dozens of sites badly, I’ll show you how to keep things on track, how to plan for success, and how to cope with failure.

 


Slides

Video

Transcription

ELLIOT:    We will begin our final talk in track B before lunch.  So lunch starts at 12.30 and if you enjoy queuing, that is the time to go if you don’t enjoy queuing wait for an hour or so.  Don’t go at 12.30 if you can because it holds everything up.  So other than that we have got this talk which will finish at just about 12.30 and we have Edd Hurst from Brighton and we are going to use this opportunity to do a plug for WordCamp Brighton which Edd is involved in as well.  So WordCamp Brighton on 18th August.  It is really good, our second one, a success last year.  Come down to the seaside it will be good fun.  So Edd works for pragmatic and has been working in WordPress for ten years or so he is also a film maker and documentary maker and he has helped rescue a whale.  That is the information I got from Edd when I asked him to talk about himself.  So I will hand over to Edd.

EDD HURST:  Thank you very much.  Microphone works, everything is good.  So my name is Edd.  I live in Brighton where I work as a producer at Pragmatic.  And for all intense and purposes this means I spend a lot of my time thinking about websites, piecing together client requirement and how to best deliver them.  I have been thinking about migrating websites and more specifically migrating several websites into one.  Sometimes I like to take work home with me and I like to think about other things and recently I have been thinking about moving house.  At some point I melt the two together and end up with a wonderful talk about how migrating content is a bit like moving house.  So moving house absolutely sucks.  No matter how many times you do it or how many things you think you will be able to do better next time or after having done it badly the time before.  Moving house is always an incredible waste of time.  It is not to say it is not worthwhile or exciting.  It brings with it opportunities and communities and extra space.  It is hard to forget the process take so long to actually do.  But one you have done it a few time it is easy to get into a rhythm and you can pick up tricks to make sure it is done better next time.  So bagging up the shopping at the supermarket putting the eggs at the top.  So no matter how many times you move each time is always different from the last that is true, we will be probably be moving most of the same things we moved before.  There is probably a whole pile of stuff you never really unpacked last time and they end up back in the attic again.  And depending on how many birthdays and iPhone cycles, depends on how much more stuff you are going to bring with you.  The house we are going to move to is probably not the House we prove to last time.  That is the key difference.  We can’t just assume that everything is going to fit.  The amount of stuff we have to take with us and the complexity helps to decide whether or not it is something we are willing to do ourselves or if we get an expert into help.  In the same way migrating website really comes down to the content itself.  How much there is, how inter connected it is, how important it is.  I am sure I can probably hire a man in a van to move across the street, 250 feet from here to about the end of this corridor.  But on the face of it it seems like he might just laugh at me.  So I can probably just do it myself.  I can see it from here.  I can probably just pick it up and a human chain gang like this table and move it around.  It is entirely likely something will break on the way and everybody will be sad.  There is a definite tipping point where there that bit too much to do yourself without potentially losing something on the way.  Equally unless you are just moving or migrating website hosts it is likely your website is being updated or even rebuilt completely, at which point the question of where it is all going to go is vital.

So I was thinking about this whilst unpacking some boxes as I said earlier I have got a ton of boxes I have not unpacked since seven month ago, so they are probably fine where they are, I think about this instead.  So I think that migrating websites shares a few similarities with moving house.  This is pretty exciting.  It is something that everybody can understand and relate to.  In my job I use a lot of metaphors in fact every they I am speaking to somebody somewhere making some kind of reference or analogy to help better under the situation and give everybody common ground to discuss what is going on.  Your website is a car.  Your hosting is the garage you are going to park it in.  An I frame is a window into somebody else’s living room you can’t change the curtains but you can watch.  Migrating is a lot like moving house.  So why is it important?  What can we learn from this?  The bulk of us have probably moved house a lot more times than we have migrated websites.  Which means that having something else you are familiar with makes it less daunting the next time you do it and discuss it.  Moving house is also something you are deeply involved with, it is all of your stuff.  You need to pack it, you need to throw some of it away, break something, everything is sad and it is tiring and memorable.  Migrating your website is not something you are always so deeply involved in.  You don’t have that same connection it happens in the background which makes it harder to remember and harder to do again.  So let’s draw some lines between the two.  Step one.  It is essential to find out more of what we can about what we are doing first.  You would not think to pick up a wardrobe without checking to see if anything is inside of that wardrobe.  And if you are forward thinking you will think about getting the therapy measure out first to make sure it fits when it gets there.  Having more information it naturally leads to us being having more informed decisions which saves a lot of wasted time getting things wrong but allows us to plan effectively.  The information does not need to be overly specific.  That being said more detailed it can be the better.  So we can start with this.  General content pages.  Core blog and WooCommerce and PDF download.  We can think maybe we don’t need some of this other functionality.  In order to get this information WordPress can help us out quite a lot total post counts make it in relatively straightforward to breakout a comprehensive report.  We have 750 posts.  No comments and 50 pages.  It changes the scale.  If we dig a little further a bit more and the list becomes a bit more concise.  Precise.  50 pages four templates.  778 blog posts 70 users.  And images forgotten about as you go along.  So first of all it is looking a bit bigger than we originally anticipated.  This is not always a problem but it is worth scheduling some time in.

So we know what we have to move.  Nice, lets just move it right?  But, not quite.  Step 2.  Set fire to everything.  Maybe not quite so dramatically.  The idea is to imagine what it would be like if you lost everything you just planned out.  What would the world look like if you did not have that blog post or three piece suite?  What would cause you the most problems to do again.  Inversely what are you glad to be rid of, that ugly painting or that sofa, the pages the that the intern post that did not work out?  All those comments that were spam.  Unlike moving house you can completely ignore some things rather than moving them or controlling them away.  They will get deleted eventually but in the meantime problem solved.  For everything else make a note of what you want to keep.  And then carry on step 3.  Where does it all go?  We have got that idea of what we want to keep.  Where is it going to go?  This is one of the first things we do when we move into a new house, walk around the rooms.  Allocate some, and quibble over future furniture decisions, some things are obvious.  The kitchen is probably the room with the cupboards in and your fridge is probably going to go there.  The personal collection of miniature hotel shampoos will go wherever the shower is.  What about everything else?  What is left may not actually be that obvious.  And if you don’t want Java you should remind me later.  Sorry.  What is left might not be that obvious.  If you are in the process of redesigning viewer site it maybe you need to allocate the place and design for this content.  Another could reason to do this early.  Ah no, we forgot about the reviews.  We need to make sure there is a star rating next to each location.  Making sure to map all of this information correctly both in the back end and the front end is pivotal to ensure you have got everything.  But it is also a great opportunity to realise if there is anything you don’t have.  This means it might mean revising a wire frame that is overly ambitious or committing to filling the gaps.  Knowing where the gaps are early can make sure everybody knows what to expect.  Step 4.  It won’t all fit.  Inversely what if you have too much stuff.  The wire frame does not allow for it, the designs don’t measure up.  We will come back to this.  We should get rid of everything else.  We have already put something in the right location.  Everything you have got left over is probably a waste of space.  What if you are moving into a one bed flat and you don’t have room for your three piece suite any more and you are moving in with everybody else and you don’t have a need for two sets of dinner plates.  If you and your website are moving to the next level it maybe that you don’t actually need all the same things any more.  Your old blog might not fit with the your new corporate message, you might not need all those products any more.  You don’t sell them.  For everything that does not have an obvious place to call home it is worth readdressing whether or not you really need it.  If it is not going to make it to the front end of the website it does not make the cut.  You are unlikely to come back in six months time and find the need for it, so get rid of it.  So how do you do it?  Well every time I have moved house I have decided this house move is going to be the one where I get it right.  And I catalogue my CDs and adequately index all of my books.  It will be great.  I download a bar code scanner and a decimal system only I understand.  I will write out what each box contains.  I will give it a number I will never lose anything again.  In fairness that is a cracking idea.  But, I am probably not the first person to think of it, but an hour later when I have only backed one box I am kind of getting fed up, shove everything in a box where it kind of fits and hoping I don’t lose anything along the way.  As such it does not really matter how you actually get that information and collate it all together.  In fact what you will probably find is that halfway through you will change it and have multiple different versions of similar data.  The important thing that whatever the method it is easy to decode ideally by more people than just yourself.  So far it looks like we are doing well.  We have itemised and trimmed the fat.  We have mapped it out so everything has a place to go.  We have informed the relevant people designers can design and developers can work out the best way to store information and how they can put the page in the right place.  Project managers are happy, everything has a plan.  As a bonus we can get rid of everything extra that does not make sense to be there any more and see the potential gaps and amend the designs or add new content to make it work.  So far so good.  Now we just need to actually do it.  So it is entirely likely that there is probably a big delay between having everything ready and actually migrating the content.  The website needs to get built and designs need to get signed off.  It is probably not the next thing you do.  Why it is important you plan it properly so it is easy to understand in six months time when you come back to it it is there ready for you to pick back up.  In some cases it can make sense to do an initial migration at some point in this process.  The developers can look in real content depending how the content is moving across and what your site is going to look like.  It probably will not be final.  The old site is still fine the stock quantities are changing, but it is a good start.  It is important to note if you do that you are going to do all of the next steps twice.  That is not necessarily a bad thing, it is important to keep in mind.  So, step six pack it up, move it out depend on the project and the requirement it maybe it needs a trial run or the real deal.  Either way the process is more or less the same.  Dry runs aren’t normally a part of a typical house move.  But, if you consider when a prime minister or president changes office to get the old one out and the new one is just a few hours while they go to a signing meeting.  I like to think of this as a scene in a movie where they build a fake vote and practice it a few times.  If you have a live website.  Specially a shop you may move things across slowly or in phases.  Once to get content in place and again when you actually go live.  So how do we do it.  There are a few options.  You can clone it.  By this effectively mean you install a plugin that will clone your website into one place to another.  It can work as a starting point and easily go in and delete or amend.  If it is just the design changing being able to dump the existing content in your new place.  Moving from A to B can be the quickest method of getting there.  You could do it yourself, copy and paste style.  Probably the least practical method available.  In some instances and I am sure many people in this room have done it it is the easiest method and how some people do it.  It is going back to hiring a man in a van to move across the road.  There are still sites out there that have hard coded contact or not easy to get content out.  There is a point which it is properly quicker to copy and paste it.  Grab it from the front end and stick it in the back end.  This is true even when you finish migrating as well.  It maybe you successfully client everything you need to move things around or add something extra.  Full disclosure.  This is not fun.  However all of the mapping and planning will make it easier to plan this out, either way it is absolutely worth doing some of this yourself just to start with.  If you can do a half hour you will work out a nice rhythm and a good idea how long it takes to do a single item and be able to plan and you will also have got wrong and you will undo it and change it and after half an hour you will have a pretty good idea what to do it.  It is any at that point I would consider hiring somebody else to do the rest unless you spent the time up front generally speaking it will all go wrong and you will spend a lot longer doing it again.  Or they will or you will and tempers will be fraught.  I once migrated 700 products by hand from a HTML website to WooCommerce.  It took two weeks and drove everybody involved crazy.  For the sanity of your workforce work that out before you start otherwise you will end up with three different solutions to every problem and it is all just gone wrong.  So the next one you can do it yourself you can use a plugin.  There is a point at which it is worth getting a little help.  In moving terms this plugin is a man with a van.  He will help you pack it up and shove it in the van and help you unpack at other end.  Much like the man with a van, different options and key skills.  All of them have their own little quirks so it is best to test it to make sure it does do what you need.  The big benefit using a plugin is it does not really make too much difference as to the scale of your data set.  You could have seven posts or several hundred, the method is pretty much the same.  There are still constraints here, it maybe it does not do it in whatever way you want it.  The bigger the differences between your old and new site structure the more complicated to export and import.  At that point it might be worth looking to do a migration expert to help out.  Even if only just four particular part of the migration process to give you technical cover.  Taking us to step 4.  Professional migration company.  When it gets that bit too complicated or more time than you are willing to put into it yourself always consider hiring a professional through raise any red flags you might have missed.  In-house moving terms the profession people that come in and pack and take it away and unpack at other end.  Everything is done for you, great, as part of the process they will be able to do that mapping and set some time aside for try runs and that all important final run.  As to how they will do that, there is no particular magic trick, they probably do the same things in other three steps.  With the potential of amending anything themselves that is a little bit tricky to do as a plugin.  Behind the scenes bit is vital.  Specially for those complicated migrations where the website is switching content management systems to merge multiple into a single website.  Sometimes spending time up front to write code is worth it.  Specially if you means you can reuse it without having to do it all again.  So how do we know which option to take.  How do we know if we can n it ourselves?  For many of us the problem of moving house is a relatively easy way to fudge your way through.  Worse case scenario shove things inboxes and drive to your destination.  The comparison breaks a little bit, moving house it is pretty easy to see what is left behind.  If that big box of old jumpers is still there rather than in the van you know something has gone wrong.  But with migrating content you properly will not have noticed it is missing until you have pact half of it.  Imagine you were moving an entire office or library or museum how about moving the entirety of WordPress.com or wikipedia with 40 million articles.  It is easy to forget that not all websites are the same size.  Many of them will share a lot of the same plugins or architecture.  The content is different.  The amount of content is different.  That amount idea of scale to not always get picked up on at the beginning which is why it is important to do those first steps and final steps, all of the steps and work out what you have got and how big it really is.  Whichever way you choose to do your migration is not just a case of working out where the content should go but when it should be put there as well.

Advice, I am sure we have all been given at one point or another is to pack your kettle last, so as soon as you arrive at your new house you can take it out and get it set up and have a nice cup of tea.  Imagine what you will say if you put the jam and cream the wrong way round.  In much the same way the order you choose to import your content can have really big consequences.  Some things will directly rely on other content being there already in order for to it work correctly.  This is all the more important if you are using an automated tool it does not know how to top and check if something is missing it assumes you know what you are doing and everything is in place.  It may not be immediately obvious as to why it has not worked as a result.  In this scenario you should not be afraid to start again.  Doing it may seem counter productive, but probably easier than fixing the problems one by one.  Often it can only be clear after having done it wrong two or three times before and realising what is missing.  This will always be a faff, having done it wrong will help you understand what you need to do to get it right and it will give you that connection to the content what you have always wanted.  So you have gone true this process.  You have got everything moved.  What does success look like?  With migration success is often mostly just recognise by it not being failure.  Sometimes this is a good thing.  I know somebody who moved into a old hotel few years ago, it had been broken into a few different houses, he happened to get the large reception area.  One day whilst making lunch an elderly gentleman walked in and asked for a glass of whisky.  Walked pass the miss mash of shoes and into an obviously domestic kitchen and failed to spot any of it.  All he knew was it was lunch time.  20 years ago he used to go to the hotel for a quick drunk at lunch.  Users are creatures of habit.  The ideal situation here is to have the user come back the next day and not be missing anything.  It might look different put all the pages they came to see are still there.  The product they wanted to buy yesterday is still available.  In some instances the customer might not realise anything has changed at all.  That is a good thing.  Not for the guy in my story, he did not get a drink, he was asked to leave.

Cool.  So those are effectively all the steps.  So what are common things you might need to keep in mind that typically go wrong or get missed or skipped along the way.  But images are a major problem and just a massive faff.  When importing your images WordPress will import them as if you were uploading them in the media library today.  Irrespective of when you added it previously all of your images will now be into they day by date folder.  So losing all of your image ranking is easily overlooked.  The most practical fix at this point is to resort to FtP or demand life to move it in its entirety to the new location.  The folder structure the same.  Google is happy.  We have also got the question of do we carry clone or migrate.  There are a lot of plugins out there you can help use migrate your website.  In my experience many of them are more like website clones than migrators.  This nice put cloning a website is not really what I am after 90% of the time.  Merging data is probably a bit more accurate.  The difference I am really looking for is being able to isolate specific types of content, posts or products or comments and move them form A to B without messing up anything else.  Things like post ideas so that everything continues to work.  A good example of migrating WooCommerce products and orders making sure the product ID of what was actually bought relates to what is in your new website now.  It can get pretty messed up quickly and it is easy to miss that it has gone wrong.  Post ID changes is another common problem.  When you migrate your content from A to B it is likely that B has some posts already.  If you are not cloning and just migrating all of your post IDs are different you will already have up loaded images and content, the new content might not relate to the right place.  That is key.  That is kind of where you get your professional migration expert to come and help map those back together even if that is all they did.  The last one is 301 redirects.  If you have been there you know how big a pain in the arse that is.  But if not come and see me afterwards and we will have a great conversation.

So much like moving house migrating a website has a lot of moving parts to it.  Don’t under estimate the challenges and hopefully by following these steps your next migration will be a happy migration.  Thank you (Applause).

ELLIOT:  Thanks Edd for that talk, we have five or ten minutes for questions, if you would like to ask one pop your hand in the air and we will whiz over with the a microphone.  Someone must be questioning today?  No questions.  Over here.

FROM THE FLOOR:  Thanks for the talk Edd.  It is all about planning to minimise the down time right if your are migrating from one place to another?  What sort of thing do you put in place to make sure you do it as quickly as possible I guess?

EDD HURST:  The plan is the most important part everything mapped together and that process in mind.  If you are doing a big migration down time is not necessarily a bad thing.  Especially if you have got a shop it is absolutely the right thing to do, the last thing you want is to have somebody buy something and mess everything up.  But, without live data changes actually just having a content freeze from the client’s perspective is probably just fine.  If you are changing your website you have properly got it is a staging environment first and you are putting that live at some point.  A content freeze from your client’s perspective you can just swap that over, nobody is the wiser.  A question there.

FROM THE FLOOR:  Thanks for the talk.  You mentioned using plugins and I just wondered if you can mention a few that might be useful.

EDD HURST:  Yes some hosts have a plugin that can help you pull a plugin, pull a site in.  And those are quite useful.  Alternatively you can use WP migrate DP pro is good for cloning a website in and making sure you pull it in.  My personal favourite WP all import.  That does that fine level moving of data without over writing on the other side and it allows you to map various things like the metastructure and adding additional fields, it can help you remember where your content comes from even if that is not being added.  That is something I forgot, it can be really useful to add extra fields in place to what the legacy URL was or post ID was so when you are looking back at it you can actually find it pretty quickly, or you can very easily map all of those IDs to the legacy ID rather than the new post ID and it helps tie that together a little bit more.

ELLIOT:  Any more questions for Edd?  Okay thanks very much Edd.  Great talk.  Appreciate that (Applause). I just want to say before lunch a big thanks to our captioning team and the AV guys and audio Guys in yellow, we have our graduates bar to talk about WordPress and any help with problems.  Other than that it is lunch now if you head on through.  Try not to rush in now because everyone is going to be rushing and once you get your lunch you are welcome to eat it throughout the area, so do spread out and make space for everyone.  We are back here in track B at two o’clock.

Speaker