Table of Contents
Vesuvius Projects - Google Summer of Code 2012
Export of Information about Those Missing and Found
Mentor-Provided Section
Background, Spring 2012
- A brief introduction of this idea and others is at GSoC 2012 Ideas
- IRC chats about this idea and others prior to the Google proposal deadline are at Vesuvius GSoC 2012 (Pre-Acceptance Version)
- An example was given of a triage-statistics table geolocated at a hospital, as a KML file: Export of Information about Those Missing and Found
Summer 2012
- Vesuvius project commit This is the prerequisite bug fix. Marked as “abandoned” for housekeeping as “done”, but being merged locally for future release.
Early June Discussions
Mentor Meeting
The mentors (Glenn, Greg, Lan) met 6/7/2012 to consider priorities and other aspects of the project, to provide input and feedback to Ahmad:
Chart of most initial interest: arrival rate (at triage station)
Possible UI flow:
- For a given disaster event, select type of chart.
- Specify or filter info of interest (e.g., gender/age/status/time range…), using new intermediate form.
- Perhaps display a preview before rendering chart.
- Export chart and/or its data.
Output channels for export (perhaps in declining priority):
- Download button (streamed to file)
- URL to temp or dynamic file (followup to Ahmad’s earlier IRC ideas); URL sent in email
- File sent as email attachment
- Feeds and web services
Output formats: .csv, XML, KML, Excel, and maybe JSON formats.
It was decided to look into PHP libraries to help with this, beyond the Highcharts javascript offerings (but ETL possibilities are postponed).
Module configuration:
A new stat module, /mod/stat2, would be created by Ahmad and Lan, a complete re-write of the current module, with possible new libraries. Key ideas:
- break up the code into logical units, instead of keeping everything in main.inc as in stat (v1)
- let form parameters tweak the dataset generated by SQL queries
- allow the data to be exported, not just viewed.
Project setup: Alternatives were discussed. See next item for results.
Export Project Code Configuration
Greg created a new Launchpad team (composed of Ahmad, Lan, and Greg) and a new code branchfor the Export Stat team to commit to during the summer. At NLM, a new instance of Vesuvius on Lan’s shell was set up to push/pull stat2 code against this.
Initial Work on the New Module, Reported by Ahmad June 8th
As reported to [sahana-agasti], Ahmad began refactoring the stats functions, separating the views from the actions into separate files, to improve readability of this complex code.
On the arrival rate page, he added an export button named “Send to”. The purpose is to generate the chart as an svg vector image embedded in a PDF file, to be then sent as an email attachment. Pressing “Send to” opens a JQuery UI dialog, which has a Submit button and these fields:
- name (from)
- to
- subject
- message
- svg_chart
On submit, the svg_chart field is filled with the chart svg data from Highcharts (escaped so it could be sent safely). This data is then used when generating the PDF file by calling the Highcharts export web service (using curl). The mail() function completes the job, with the attachment base64- encoded.
For more: the main PHP code, and adding the Highcharts menu item
Summary of Ahmad's June 15th Update
(To take advantage of Ahmad's PHP experience, it was decided earlier to have him redesign and rewrite the module from scratch.) Consequently, Ahmad has begun the skeleton of a “stat2” module. As part of this, he suggested and successfully tested the use of open-source HighRoller library as a PHP wrapper for the Highcharts Javascript library. It could render multiple charts per page. Ahmad edited HighRoller code to support data exporting, by calling a custom Javascript callback function: “commit_$renderTo”. This can be used to write custom responses for the “sendTo” exporting menu item.
Implementation steps in ~gsoc-stat branch, through code revision 5272:
- Created mod/stat2 (which tests HighRoller lib)
- Added '3rd/HighRoller'
- Edited .htaccess file to support the new module (replacing the rewrite rule of original stat module)
- Added a new role for the module in the ACL database table (see SQL changes in backups/stat2.sql and CHANGELOG_DB)
- Deleted newstat branch
Summary of Ahmad's July 1st Update
The main rewriting of the Vesuvius Stat module is done, based on the MCV pattern with the models, controllers, and views now in separate files. This will facilitate the export of data by web service. Using the highRoller PHP library as a wrapper for the Highcharts Javascript library improves code brevity and and maintainability.
Planned: to add filtering functionality, and start building the web service methods to export the statistics module data, for use by client applications.
Implementation steps in ~gsoc-stat branch, through code revision 5274:
- Created /mod/stat2/libs folder - contains the models of the stat module
- Created /mod/stat2/views folder - contains the views of the stat module
- Main.inc - created actions for the different charts
- 3rd/highRoller/highRoller.php - added new methods
IRC Chats
These were on #sahana-meeting unless indicated otherwise. Times shown may be full chat or most-significant portion.
- Tuesday, May 15, 16:07-17:20 UTC (12:07-1:20 pm EDT). This was a general Agasti meeting but much of it concerned GSoC
- Thursday, May 31, 2012, 18:02-19:25 UTC (2:02-3:25 pm EDT)
- Tuesday, June 12, 16:04-16:59 UTC (12:04-12:59 EDT).
- Tuesday, June 19, 16:05-16:59 UTC (12:05-12:59 EDT).
- Tuesday, July 3, 16:01-17:04 UTC (12:01-13:04 pm EDT)
Student-Provided Section
Abstract:
The new Vesuvius “Stats” module displays charts with timeline and cumulative statistics. This includes overviews of missing and found people and their reported attributes such as health status or location. We would like to extend the module to offer the underlying cumulative or de-identified data in various formats (e.g., XML, JSON, csv, Excel) and through various channels (e.g., SOAP and/or REST web services, rss/atom data feeds). Of particular interest are GIS formats, with geolocated data (e.g., KML, GeoRSS) suitable for posting to disaster-related globes & maps. This would aid situational awareness and understanding of missing person spatial distributions and hospital triage-station impacts.
Code by Topic
module init
http://bazaar.launchpad.net/~gsoc-stat/vesuvius/trunk/revision/5271
http://bazaar.launchpad.net/~gsoc-stat/vesuvius/trunk/revision/5272
Rewriting Module
http://bazaar.launchpad.net/~gsoc-stat/vesuvius/trunk/revision/5273
http://bazaar.launchpad.net/~gsoc-stat/vesuvius/trunk/revision/5274
Adding Date Range Filter and OOP modifications
http://bazaar.launchpad.net/~gsoc-stat/vesuvius/trunk/revision/5275
http://bazaar.launchpad.net/~gsoc-stat/vesuvius/trunk/revision/5276
http://bazaar.launchpad.net/~gsoc-stat/vesuvius/trunk/revision/5277
http://bazaar.launchpad.net/~gsoc-stat/vesuvius/trunk/revision/5278
Adding Export Functionality
http://bazaar.launchpad.net/~gsoc-stat/vesuvius/trunk/revision/5303
http://bazaar.launchpad.net/~gsoc-stat/vesuvius/trunk/revision/5304
http://bazaar.launchpad.net/~gsoc-stat/vesuvius/trunk/revision/5305
http://bazaar.launchpad.net/~gsoc-stat/vesuvius/trunk/revision/5306
http://bazaar.launchpad.net/~gsoc-stat/vesuvius/trunk/revision/5307