Vesuvius Projects - Google Summer of Code 2012

Export of Information about Those Missing and Found

Mentor-Provided Section

Background, Spring 2012

Summer 2012

  • Vesuvius project commit This is the prerequisite bug fix. Marked as “abandoned” for housekeeping as “done”, but being merged locally for future release.
Early June Discussions

Mentor Meeting

The mentors (Glenn, Greg, Lan) met 6/7/2012 to consider priorities and other aspects of the project, to provide input and feedback to Ahmad:

Chart of most initial interest: arrival rate (at triage station)
Possible UI flow:

  1. For a given disaster event, select type of chart.
  2. Specify or filter info of interest (e.g., gender/age/status/time range…), using new intermediate form.
  3. Perhaps display a preview before rendering chart.
  4. Export chart and/or its data.

Output channels for export (perhaps in declining priority):

  • Download button (streamed to file)
  • URL to temp or dynamic file (followup to Ahmad’s earlier IRC ideas); URL sent in email
  • File sent as email attachment
  • Feeds and web services

Output formats: .csv, XML, KML, Excel, and maybe JSON formats.
It was decided to look into PHP libraries to help with this, beyond the Highcharts javascript offerings (but ETL possibilities are postponed).

Module configuration: A new stat module, /mod/stat2, would be created by Ahmad and Lan, a complete re-write of the current module, with possible new libraries. Key ideas:

  • break up the code into logical units, instead of keeping everything in as in stat (v1)
  • let form parameters tweak the dataset generated by SQL queries
  • allow the data to be exported, not just viewed.

Project setup: Alternatives were discussed. See next item for results.

Export Project Code Configuration

Greg created a new Launchpad team (composed of Ahmad, Lan, and Greg) and a new code branchfor the Export Stat team to commit to during the summer. At NLM, a new instance of Vesuvius on Lan’s shell was set up to push/pull stat2 code against this.

Initial Work on the New Module, Reported by Ahmad June 8th

As reported to [sahana-agasti], Ahmad began refactoring the stats functions, separating the views from the actions into separate files, to improve readability of this complex code.

On the arrival rate page, he added an export button named “Send to”. The purpose is to generate the chart as an svg vector image embedded in a PDF file, to be then sent as an email attachment. Pressing “Send to” opens a JQuery UI dialog, which has a Submit button and these fields:

  • name (from)
  • to
  • subject
  • message
  • svg_chart

On submit, the svg_chart field is filled with the chart svg data from Highcharts (escaped so it could be sent safely). This data is then used when generating the PDF file by calling the Highcharts export web service (using curl). The mail() function completes the job, with the attachment base64- encoded.

For more: the main PHP code, and adding the Highcharts menu item

Summary of Ahmad's June 15th Update

(To take advantage of Ahmad's PHP experience, it was decided earlier to have him redesign and rewrite the module from scratch.) Consequently, Ahmad has begun the skeleton of a “stat2” module. As part of this, he suggested and successfully tested the use of open-source HighRoller library as a PHP wrapper for the Highcharts Javascript library. It could render multiple charts per page. Ahmad edited HighRoller code to support data exporting, by calling a custom Javascript callback function: “commit_$renderTo”. This can be used to write custom responses for the “sendTo” exporting menu item.

Implementation steps in ~gsoc-stat branch, through code revision 5272:

  • Created mod/stat2 (which tests HighRoller lib)
  • Added '3rd/HighRoller'
  • Edited .htaccess file to support the new module (replacing the rewrite rule of original stat module)
  • Added a new role for the module in the ACL database table (see SQL changes in backups/stat2.sql and CHANGELOG_DB)
  • Deleted newstat branch

Summary of Ahmad's July 1st Update

The main rewriting of the Vesuvius Stat module is done, based on the MCV pattern with the models, controllers, and views now in separate files. This will facilitate the export of data by web service. Using the highRoller PHP library as a wrapper for the Highcharts Javascript library improves code brevity and and maintainability.

Planned: to add filtering functionality, and start building the web service methods to export the statistics module data, for use by client applications.

Implementation steps in ~gsoc-stat branch, through code revision 5274:

  • Created /mod/stat2/libs folder - contains the models of the stat module
  • Created /mod/stat2/views folder - contains the views of the stat module
  • - created actions for the different charts
  • 3rd/highRoller/highRoller.php - added new methods
IRC Chats

These were on #sahana-meeting unless indicated otherwise. Times shown may be full chat or most-significant portion.

Student-Provided Section


The new Vesuvius “Stats” module displays charts with timeline and cumulative statistics. 
This includes overviews of missing and found people and their reported attributes such as
 health status or location. We would like to extend the module to offer the underlying 
cumulative or de-identified data in various formats (e.g., XML, JSON, csv, Excel) and 
through various channels (e.g., SOAP and/or REST web services, rss/atom data feeds). 
Of particular interest are GIS formats, with geolocated data (e.g., KML, GeoRSS)
 suitable for posting to disaster-related globes & maps. This would aid situational
 awareness and understanding of missing person spatial distributions and hospital
 triage-station impacts.

Code on Launchpad

Code by Topic

module init

Rewriting Module

Adding Date Range Filter and OOP modifications

Adding Export Functionality


QR Code
QR Code agasti:vesuvius:gsoc2012:export (generated for current page)