Incremental Migration of Error Items Only & Retry
The idea is to use the migration report to start a new migration of the files that error out only, instead of running a full incremental migration.
Sarah
shared this idea
As much as we really like this idea, it doesn’t fit in the current narrative of ShareGate Desktop.
We will for sure reevaluate this one in a near future.
19 comments
-
abdallah.marrouche
commented
I see this (in addition to recycle bin) a MUST. When you have tens of Gigabytes and more than thousand files fails, you just need to retry the whole thing again and will fail again.
This is a Must, please!
-
DS
commented
+100 to Anikke - this is painful finding these errors and redoing the entire migration is definitely overkill and can actually introduce errors.
-
Anikke Bukowski
commented
This is a real shame, particularly after the initial response from @Natalie. Having to redo a migration in its entirety, or trying to manually find the items that have errored by their ID to remigrate them is time-consuming and painful. Is there a technical blocker for this feature?
-
Aaron
commented
Frankly, I'm very surprised at the Admin response of "it doesn’t fit in the current narrative of ShareGate Desktop". I would consider addressing Sharegate Desktop migration failures as one of the top priorities, exceeded only by more reliable initial migrations.
We routinely encounter issues with migrations. Some may not copy because of a timeout or throttling issue. Some may incorrectly report access denied, but work successfully on subsequent attempts. There may be some errors that will require changing specific settings to address the issue. However, the bulk of the errors can be addressed by re-migrating the failed content in Normal mode with Overwrite specified. Without Overwrite, another incremental may not address the problem, since the time stamp of a partial copy will likely be the same as the source, or a newer timestamp from when the copy was performed.
To handle these errors, I've resorted to the following, mostly using PowerShell. I programmatically export all of my migration logs as CSV files. I programmatically filter the exported log files, to look for specific Error and Warning event types, and dump that subset of events into other files (FileErrors, SiteErrors, and OtherErrors). I manually review the CSV files to identify rows where fields like Warnings, Errors, or Details have exceeded reasonable limits and wrap for hundreds of lines (It would be great if export results were cleaner as well). I clean up that wrapped data for better readability. I re-migrate SiteErrors using the same scripts that were used for the initial site migration. I have a different set of scripts to re-migrate files that threw errors. There will also be some errors from re-migrating FileErrors that may require changes like limiting versions, but the number of errors that require manual intervention is significantly reduced. The OtherErrors are errors that need further review and may require manual migration. I have also used processes for handling Folder errors as well, but more often than not, I’ve found those to be reported inaccurately as a part of poor OneNote migration support. Of course, the errors which need to be addressed depends on which version of Sharegate you are running, and what has been changed in that version.
Using custom scripts and complicated processes on my end to handle Shargate shortcomings is less than ideal. Changes made in Sharegate application development can/will adversely affect my processes, which is why the Sharegate Desktop is the ideal location to be addressing the issues. If content migration reliability can’t be increased, than ease of addressing errors is of the upmost importance.
-
jmongell
commented
When a long running migration errors on a handful of checkout docs, why do I need to rerun the entire migration again?
I would love to be able to import a previously exported error (or warning) list and retry. -
Anonymous
commented
When you do an import taks from a file share there will be files that cannot be copied / imported. Maybe it is a good idea to be able to rerun the same task just for the failed items ? I had just run a task that took 5 days to complete and after that I ran the same task again in incremental mode but that also took 5 days ??
-
Cheryl Poon
commented
I don't think this is currently possible but an awesome feature would be to be able to click on an entry in the results screen after a transfer and retry that line. The breadcrumb trail at the top of the page calls it the Migration Report. The reason this would be useful is that on a large migration, there may be just a small subset of files that failed for one reason or another or sometimes it looks like it may have just been a temporary network issue. If we have fixed the issue or just want to send it through again and see if it will go through on a second attempt, we have to either send the whole large migration through again (this one I am working on took hours to process) or we have to track down where each file is located and try to push each one through again. If I could start from the Migration Report, sort on the Status so the Errors list at the top, and act on the item from that listing, that would be great.
Thanks,
Cheryl Poon -
Annette Boyd
commented
This would be a FANTASTIC idea. I'm struggling right now with moving a doc library with hundreds and hundreds of documents, and it would be FANTASTIC if I could grab just those that with an error (list item on lookup not found) and could just re-run those items.
-
Anonymous
commented
Being able to create a re-run of just the items that resulted in an error would be great!.
e.g. just tried using duplicate site collection resulting in 400+ error. (When I do this via a migration I only get ~20)
Things like Page layout could not be found could be corrected in sharepoint quite easily (assuming the layout and content types were successfullly migrated). Select the items that errored and re-run.
Thanks
-
Anonymous
commented
It'd be a significant time saver if there was a way to re-run a migration job on only those items that errored. We get random 407 errors and always have to do a full incremental run which still is wasted cycles when we only need to retry a few dozen records out of thousands.
-
Marc D Anderson
commented
When we're doing a Bulk Update of metadata (one of the most awesome uses of Sharegate!), sometimes we run into errors when documents are checked out. It would be great if we could do a retry of just those updates after we've checked in the docs.
I would envision this being useful in other cases as well. Bascially, set the Filter to show the errors, go off and fix whatever the issues are (often requireing some sort of manual intervention), and then apply only the changes which have errored out.
-
Anonymous
commented
Occassionally I will see Error 500 errors or connection errors, it would be amazing if there was a right-click retry option for errors. That way you can sort on Error - Retry all and what ever fails the second time you could investigate.
-
anikke.bukowski
commented
A retry feature would be great. Often, errors in migration result from previous versions of items having entries that no longer exist in choice fields, and the choice field not being set to allow alternative entries. It would be great to be able to make the change to the list, then retry the items that have thrown an error.
-
esteban.salinas
commented
Would be nice to have a feature to be able to retry to copy a file from the error log.
-
Anonymous
commented
I am copying subsites from on-premises to Office365 and somewere copied over and yet people don't want them! I know, a waste! So I am going through and deleting on the Office365 side with the great right click feature to delete the site, when I do, rarely I get a comunication error and I wonder if you can set it to retry if it failed, instead of going and clicking and right clicking and deleting again? Thank you!
-
jmongell
commented
Upvoting this feature request. Woudl be a timesaver in large migrations.
-
rpearson
commented
Agree with this need/functionality; to be able to act on reported migration errors.
Another example is when moving content. My steps have been to select a set of items or documents from source (like a date range) and copy to destination, when copy is complete, delete the selected items from source. When there are errors, the only recourse has been to export the errors to Excel, then find each of the error items in the source selection and de-select each error. After those error items are located and de-selected, I can delete the remaining selection of source items. Then finally deal with the error items... Quite complex and time consuming.
-
Sarah
commented
I want to run a copy from the report results. Or maybe get the option to export the report in Excel in a format that would allow me to do a content copy from that.
-
Anonymous
commented
When a list of errors is presented, it would be really handy to have a "Retry" button for each error. I have tried to run a report which has produced errors on a lot of site collections. If I try to fix the error by adding my account as a site collection administrator, I want to test it's worked before doing all the others. It would be handy to retry just the one site collection.