Why work harder when you can work smarter? Work on your data in your desktop GIS or spreadsheet app, and magically, your web maps just.. update.
Data Sync is available on the Enterprise and Agency Plans.
Data Sync currently supports standard Dropbox accounts
- Dropbox Business accounts are not currently supported by Mango Data Sync.
- If you'd like to see Dropbox Business supported, register your interest by emailing support@mangomap.com
Data Sync lets you do complex work on datasets in your desktop GIS or spreadsheet application, and have changes automatically updated on your webmaps, without even logging into Mango.
Once a dataset is synced with Mango, we listen to Dropbox for changes to the dataset, and automatically pull the updated files into Mango, process the changes, and hey presto, all maps using that dataset will be updated.
This means that quantity and category visualizations will update, generating new visualizations; the data in your attribute popups will update; new features will draw on your maps.
Data sync lets you get on with what's important.
How to configure Data Sync
If you don't have a Dropbox account, sign up for one here.
- If the dataset you want to sync is not already in your Mango account, upload the dataset to Mango.
- Upload the same version of the dataset to a folder in your Dropbox.
- In Mango, open the dataset you want to sync from your Data Portal.
- Click on DATA SYNC on the sidebar
- Click on the Dropbox icon and authorise Mango Data Sync to access your files*
- Select all files that make up the dataset from your Dropbox, then click Save.
Mango will process the file as a re-upload to ensure the file can be synced. Once completed, the sync will trigger when Dropbox notifies Mango the file(s) have been updated.
You can monitor the status of each synced dataset by opening the Data Sync panel from your Data portal.
Sync frequency
Data syncs are queued to update at a minimum interval of 1 hour, and only process a reupload when no changes to a dataset are detected during the queuing process. This ensures there are no concurrency and versioning issues for large multi-file datasets.
If a dataset currently in the queue is updated in Dropbox, it will return to the end of the queue.
If your dataset is being updating at a very high frequency, such as every 5 to 10 minutes, 24 hours a day, your dataset will never be processed as a reupload. We recommend updating datasets in your Dropbox no more than once every 3 - 4 hours to ensure your dataset passes the Sync queue.
During peak periods, the sync queue may be longer to accommodate processing delays.
If you have not received a sync confirmation email after 24 hours of data being updated in Dropbox, and no failure event emails have been received, please contact support.
Queue limiting
Due to the way Dropbox sends updates to Mango, Data sync defers processing via a queuing system to ensure our processing servers are not inundated with reupload requests. If you have a script performing updates to your data in Dropbox, we recommend not updating more than 5 datasets per hour to ensure all updates are queued, and successfully reuploaded to your Mango account.
Data Sync errors
Data sync is an automated reupload, so all the reupload rules apply. If you experience a processing failure, please confirm your data does not contain issues listed in the common upload errors for the data type you have synced.
Common causes of data sync failures
- The new version has an unrecognised projection
- The new version is missing columns that exist in the version in Mango
- The new version has column names that have changed
- The original dataset has been deleted from Dropbox
- The original dataset has been moved to a different location in Dropbox
- You have revoked access for the Mango Sync app in your Dropbox account
Re-syncing after a sync failure
If your sync has failed due to a processing error, all that is required after a sync failure is to update Dropbox with a good version of the data. Mango will requeue the sync each time the data is updated in Dropbox.
If you see repeated failure notifications after updating Dropbox with good data following a sync failure, we recommend you do a manual reupload with the same version in Dropbox to ensure the dataset uploads without errors. Errors on the manual reupload will indicate issues with the datatset that must be rectified before updating the version in Dropbox version.
Important considerations
Overwrite the data in Dropbox, do not delete/move and replace
If you back-up the dataset, or update the data, do not delete or move the old dataset and create a new version. You must overwrite the existing files in the Dropbox for Mango to be able to import them. When new files are created in Dropbox they are assigned a new unique file ID. As Mango is only authorised to access the specific file ID's granted when the sync is configured, the newly created file ID’s are not recognised and will not be synced, even if it exists in the same location as the previous file and has a matching file name.
The sync is uni-directional
The sync works in one direction: Dropbox → Mango. If you make edits to the dataset in Mango, they will be overwritten by a subsequent sync if those changes are not manually integrated into the version in Dropbox.
If you are using Data Sync to update your data, we recommend that updates such as new features, feature removals, or updates to attributes and geometry are performed in external applications to avoid losing changes that may have been made in Mango.
Maintain existing fields/columns
Like all re-uploads, the synced dataset cannot contain column names that have changed name, or columns that have been removed. New columns are OK. If you deprecate data in once column, you can include a new column with new data. Remember to update your feature popups to display the new attribute variable.
Mango will not process syncs where the newer version contains columns that have been renamed or removed.
Frequency of updates
The sync requires at least 60 minutes between file updates to reach the front of the queue. If changes are being made to the file more frequently than 60 minutes, our queuing system will not complete and the reupload will not occur.
Folder structure and file limits
Ensure that no more than 500 files exist in any Dropbox folder containing Synced datasets. If more than 500 files exist in a single folder containing a synced dataset, Mango may not receive the update notifications from Dropbox, and therefore may not import the new version of the dataset.
Use lowercase file extensions
You may experience errors when setting up a data sync if you use capitalised file extensions. For example, data.CSV. We recommend always using lowercase file extensions: data.csv.
Multiple Dropbox accounts
Dropbox authorisation is stored at the dataset level in Mango. This means that any user can configure Data Sync on a dataset provided they have edit permission on the dataset, and any number of Dropbox accounts can be used to synchronise datasets.
However, for the sake of simplicity in managing your synced datasets, we recommend only one account is used.
If you update the location of an active sync and authorise a new Dropbox account for a dataset, any changes to the dataset in the previous Dropbox account will fail to update the data in Mango.
Updating files in Dropbox should be performed by the same Dropbox user that configured the sync for that dataset.
Revoking Dropbox authorisation
If you stop using Data Sync, you should revoke the Mango Sync app in your Dropbox account.
To revoke access, sign in to your Dropbox account, navigate to Settings → Connected apps → Linked apps, and disconnect Mango Sync
Supported browsers
To work with Data Sync, your browser must support dropbox.com. Visit Dropbox to learn more.
Data Sync may not be fully supported on older versions of some browsers, or on browsers not listed at the above link. If you experience issues authenticating Dropbox with Mango Data Sync, please check you are using a supported browser.
Privacy
Mango Data Sync only has permissions to access files and folders that you explicitly grant during the Sync process. No other files or folders in your Dropbox are accessed, or accessible by Mango.