Table of Contents
Google Summer of Code 2010: Automatic Synchronization between Sahana Instances
This project is part of Sahana Eden Google Summer of Code
Executive Summary
- Abstract: Automatic sync between Sahana instances has received much attention from Sahana community. It aims to sync resources between multiple Sahana instances. I will work on resolving its current issues and adding new features:
- Modifying sync process to work with S3XRC resources instead of DB tables to maintain full references
- Offline Sync facility (USB)
- Add the facility to schedule multiple custom sync jobs (custom - weekly/daily/one-time with selected or all resources)
- Make sync peer discovery work over WANs & LANs by combining ZeroConf with Beacon Service (an online directory of Sahana instances)
- Student: Amer Tahir
- Mentor(s): Fran Boon and Michael Howden
Code
How To Use
Downloading and Installation
Do “bzr checkout lp:~amertahir/sahana-eden/sync” (without quotes) to get the latest source-code from repository. Then rename the sync directory to eden and place it under your web2py applications directory. That's all, now fire up your browser and open http://127.0.0.1:8000/eden/ to get started :)
NOTE: the demo instances (specified above) could also be used to test out sync (use username: admin@example.com and password: admin to login)
Getting Started
After setting up the instance, you have to first Register (or Login with administrative account) and specify a few settings for sync. For this, open Administration page and then click Sync > Sync Settings from the menu. You will see the settings page for sync. Here you have to specify you instance URL (e.g. http://sync.eden.sahanafoundation.org/eden), description and so on.
Then proceed to the Synchronisation main page. From there you can sync manually (click “Sync Now”), add sync partners, subscribe to sync pools, resolve sync conflicts and so on. The page is self-explanatory.
How to Sync?
Synchronisation process works automatically in the background (as Cron jobs) and you can use the Sync Scheduler to manage sync jobs. However, if you wish to bypass that and manually perform sync, you can click “Sync Now” button to start syncing with sync partners that you have specified.
Sync Partners
Sync Partners are instances that you wish to sync with. They could be deployed on SahanaEden, SahanaAgasti, Ushahidi or possibly any platform that SahanaEden can talk to (currently Eden, Agasti and Ushahidi are supported).
Before you can start syncing, you need to define sync partners. This can be done by going to the Sync Partners page. You can either search for partners or add them manually (right now only manual additions are supported). Every sync partner has a UUID (0 for non-Eden instances), instance URL (e.g. http://sync.eden.sahanafoundation.org/eden), instance type (Eden/Agasti/Ushahidi/etc.) and Sync Policy. Sync policies other than “No Sync” only come into play when the data that is imported also exists locally. It could be the case that local data is modified after the last sync with this peer (Local is newer), similar for remote data (Remote is newer) or both have been modified (either of them is newer). So, to decide how to resolve such 'conflicts' automatically, you can select one of the sync policies. If you want to manually resolve such conflicts, select “Choose Manually”.
Sync Schedules
To automate the sync process, you can define sync jobs. Sync Jobs are individual syncing processes with sync partners that you like. So, if I want to sync “Person Registry” with some sync partner, I would just create a new sync job and select “Person Registry” and the sync partner I want to sync with. Along with that, I would specify the scheduled time for the job to run, either daily, weekly, hourly or just once.
You can define multiple sync jobs for a particular sync partner, specifying resources to sync and sync policies individually for each of the sync jobs. By default, only resources modified after the last sync operation (new data) are synced. This is similar to differential backups. However, if you want to sync all data, you can specify “Complete Sync” in the job settings.
This section is a work in progress and would be updated as development is carried out on this project
Progress
Weekly Meetings
- Thursdays at 1400 UTC Calendar at IRC (this is tentative schedule, *would* change)
Progress Reports
- Every Friday on the Mailing List
Project Plan and Timeline
Following are the chosen SMART Goals that were recommended during the application process, and reviewed over the bonding period:
Midterm Evaluation Milestones: 24 May - 12 July | |||
---|---|---|---|
Goal | Measure | Due Date | Status |
Modify the existing sync process to work with S3XRC import/export instead of working directly with DB tables (and records) | Prototype demo of the S3XRC powered sync system | 06/09/2010 | Completed |
Incorporate automatic conflict resolution policies in the sync system | Demo of a working (possibly buggy) implementation | 06/20/2010 | Completed |
Finalizing conflict resolution policies after bug fixing and preparation of test scenarios to verify it | Demo of the sync system with conflict resolution including testing that would be approved after review by the mentor(s) | 06/29/2010 | Completed |
Finalize the sync system with bug fixing and changes suggested by the mentor(s) along with testing of the entire system | Demo of the sync system with testing procedures defined (Test Plan); approval by the mentor(s) | 07/11/2010 | Completed |
Final Evaluation Milestones: 13 July - 9 August | |||
---|---|---|---|
Goal | Measure | Due Date | Status |
Implementing Sync Scheduler to allow scheduling of multiple sync operations (jobs) (possibly buggy) | 07/18/2010 | Completed - late | |
Finalizing Sync scheduler, testing with the integrated system | 07/20/2010 | Completed | |
Implement the Offline Sync (USB flash sync) functionality in the sync system | Prototype demo of the offline sync | 08/01/2010 | Incomplete |
Bug fixing of the components of the sync system | Progress report submitted to the mentor(s), then reviewed and approved by them | 08/05/2010 | Completed |
Final integration of the entire system, preparing it for the demo | Demo of the entire sync system, approval by the mentor(s) | 08/09/2010 | In progress |