Incremental Migration of Error Items Only & Retry
The idea is to use the migration report to start a new migration of the files that error out only, instead of running a full incremental migration.
Sarah
shared this idea
As much as we really like this idea, it doesn’t fit in the current narrative of ShareGate Desktop.
We will for sure reevaluate this one in a near future.
26 comments
-
Anonymous
commented
It is frustrating to have to go back a screen or two to retry a move when there are only a few "warning" or "errors" that resulted from the move. It would be great if from the results screen , if there was a option to right click a line item that had an issue and click retry.
You would do this after making the appropriate changes to correct the issue of course. But having to go back and trendge through processing all of the other items (even though you click incremental). It still takes time to compare the last changed dates, which means more time waiting.
Where as if you could just kick it off right from the result page, it would make things a lot faster.
Feedback sent from the Migration report view. -
Viktor Vallin
commented
Is there any progress with this? Struggling with this almost daily..
-
fabian.heil
commented
It would be very helpful to have a better option to retry all errored items of a content migration. Otherwise you need to search for every single ID or do a huge delta copy (and that takes very long, when its a big list)
-
Anonymous
commented
the frustration is when the import portion fails on the Microsoft side. At least having those auto retry a few times would be helpful.
-
Anonymous
commented
When you have items that failed for unknown reason/threshold/import type reason. Would love to be able to grab those in the filter and manually retry. Or, have settings in job to retry 5 times. Or something similar. When we have large lists it is very difficult to hunt those down and hand pick them to retry.
-
abdallah.marrouche
commented
I see this (in addition to recycle bin) a MUST. When you have tens of Gigabytes and more than thousand files fails, you just need to retry the whole thing again and will fail again.
This is a Must, please!
-
DS
commented
+100 to Anikke - this is painful finding these errors and redoing the entire migration is definitely overkill and can actually introduce errors.
-
Anikke Bukowski
commented
This is a real shame, particularly after the initial response from @Natalie. Having to redo a migration in its entirety, or trying to manually find the items that have errored by their ID to remigrate them is time-consuming and painful. Is there a technical blocker for this feature?
-
Aaron
commented
Frankly, I'm very surprised at the Admin response of "it doesn’t fit in the current narrative of ShareGate Desktop". I would consider addressing Sharegate Desktop migration failures as one of the top priorities, exceeded only by more reliable initial migrations.
We routinely encounter issues with migrations. Some may not copy because of a timeout or throttling issue. Some may incorrectly report access denied, but work successfully on subsequent attempts. There may be some errors that will require changing specific settings to address the issue. However, the bulk of the errors can be addressed by re-migrating the failed content in Normal mode with Overwrite specified. Without Overwrite, another incremental may not address the problem, since the time stamp of a partial copy will likely be the same as the source, or a newer timestamp from when the copy was performed.
To handle these errors, I've resorted to the following, mostly using PowerShell. I programmatically export all of my migration logs as CSV files. I programmatically filter the exported log files, to look for specific Error and Warning event types, and dump that subset of events into other files (FileErrors, SiteErrors, and OtherErrors). I manually review the CSV files to identify rows where fields like Warnings, Errors, or Details have exceeded reasonable limits and wrap for hundreds of lines (It would be great if export results were cleaner as well). I clean up that wrapped data for better readability. I re-migrate SiteErrors using the same scripts that were used for the initial site migration. I have a different set of scripts to re-migrate files that threw errors. There will also be some errors from re-migrating FileErrors that may require changes like limiting versions, but the number of errors that require manual intervention is significantly reduced. The OtherErrors are errors that need further review and may require manual migration. I have also used processes for handling Folder errors as well, but more often than not, I’ve found those to be reported inaccurately as a part of poor OneNote migration support. Of course, the errors which need to be addressed depends on which version of Sharegate you are running, and what has been changed in that version.
Using custom scripts and complicated processes on my end to handle Shargate shortcomings is less than ideal. Changes made in Sharegate application development can/will adversely affect my processes, which is why the Sharegate Desktop is the ideal location to be addressing the issues. If content migration reliability can’t be increased, than ease of addressing errors is of the upmost importance.
-
jmongell
commented
When a long running migration errors on a handful of checkout docs, why do I need to rerun the entire migration again?
I would love to be able to import a previously exported error (or warning) list and retry. -
Anonymous
commented
When you do an import taks from a file share there will be files that cannot be copied / imported. Maybe it is a good idea to be able to rerun the same task just for the failed items ? I had just run a task that took 5 days to complete and after that I ran the same task again in incremental mode but that also took 5 days ??
-
Cheryl Poon
commented
I don't think this is currently possible but an awesome feature would be to be able to click on an entry in the results screen after a transfer and retry that line. The breadcrumb trail at the top of the page calls it the Migration Report. The reason this would be useful is that on a large migration, there may be just a small subset of files that failed for one reason or another or sometimes it looks like it may have just been a temporary network issue. If we have fixed the issue or just want to send it through again and see if it will go through on a second attempt, we have to either send the whole large migration through again (this one I am working on took hours to process) or we have to track down where each file is located and try to push each one through again. If I could start from the Migration Report, sort on the Status so the Errors list at the top, and act on the item from that listing, that would be great.
Thanks,
Cheryl Poon -
Annette Boyd
commented
This would be a FANTASTIC idea. I'm struggling right now with moving a doc library with hundreds and hundreds of documents, and it would be FANTASTIC if I could grab just those that with an error (list item on lookup not found) and could just re-run those items.
-
Anonymous
commented
Being able to create a re-run of just the items that resulted in an error would be great!.
e.g. just tried using duplicate site collection resulting in 400+ error. (When I do this via a migration I only get ~20)
Things like Page layout could not be found could be corrected in sharepoint quite easily (assuming the layout and content types were successfullly migrated). Select the items that errored and re-run.
Thanks
-
Anonymous
commented
It'd be a significant time saver if there was a way to re-run a migration job on only those items that errored. We get random 407 errors and always have to do a full incremental run which still is wasted cycles when we only need to retry a few dozen records out of thousands.
-
Anonymous
commented
You need to provide a method to rerun scheduled tasks that fail, esp. due to connection errors.
I have 70 Permissions Matrix reports that are scheduled to run monthly. When I have a connection failure, the only way to reconnect is to stop and start Sharegate from the system tray. This refreshes all of my connections, but I have to manually rerun all 70 of my reports, which is an extensive grueling effort.
For a tool as robust as Sharegate, THIS IS UNACCEPTABLE! -
bj.pohl
commented
For long reports, there are often communication errors. It would be great if there was a good UX way to select the containters that had errors and rerun a report just over those containers.
-
Marc D Anderson
commented
When we're doing a Bulk Update of metadata (one of the most awesome uses of Sharegate!), sometimes we run into errors when documents are checked out. It would be great if we could do a retry of just those updates after we've checked in the docs.
I would envision this being useful in other cases as well. Bascially, set the Filter to show the errors, go off and fix whatever the issues are (often requireing some sort of manual intervention), and then apply only the changes which have errored out.
-
Anonymous
commented
Occassionally I will see Error 500 errors or connection errors, it would be amazing if there was a right-click retry option for errors. That way you can sort on Error - Retry all and what ever fails the second time you could investigate.
-
anikke.bukowski
commented
A retry feature would be great. Often, errors in migration result from previous versions of items having entries that no longer exist in choice fields, and the choice field not being set to allow alternative entries. It would be great to be able to make the change to the list, then retry the items that have thrown an error.