Project Abstract

The Sahana Agasti project (the Mayon platform) has a scheme with lot of modules. Each module has other dependencies and they are connected with others, but we can build a way through web services to make available only a few desired entities of the scheme, so that another software can consume it (a different project, like Sahana Eden) in an easy way, with a simple HTTP request.

Additional Info

It is very important to identify some point of the schema which could be encapsulated within a system (simply by accessing the RESTful web service and requesting a resource). Some of the data-points of Agasti Mayon have fewer dependencies than others, and maybe those are the ideal point to develop the web service solution: the strategy therefore is to identify an easy module and then build a template - solution (to reuse in more complex modules in future). [1]

Web Services Integration

The purpose of this project is to identify and develop a standards-based web service for one of Agasti Mayon's many datapoints and to use the developed code as a template for the development of additional services after the closure of the GSoC project. While identification of which datapoint to expose is part of this project, it is strongly recommend that the exposed service either comply with existing emergency management standards such as PFIF or any of the EDXL standards and/or provide a service consumable by other Sahana Software Foundation softwares such as Eden or Vesuvius. The the selected student will work with the project mentor to identify a datapoint, design the request parameters, select the data from the Doctrine ORM models, and respond using JSON and XML, depending on the user request. Requests and responses should be RESTful and will need to be integrated with the sfGuard authentication suite to prevent unintended data exposure. This project also supports the creation of the relevant routes and application control panels, as-appropriate. Project code, at closure, should be packaged and be independently installable as a module or package. [2]

Documentation

This projects aims the construction of a web service to facilitate data interchange. Currently, the project has the web service itself already done and working; this milestone makes possible write the documentation to publish the advances and the mainly issues and to make possible the re-implementation of others web services or the extension of this current one to reach others data points.

To codify the web service, it was used a package inside the well-known structure of Agasti's code. In this package, some files deserve some highlight:

  • i) %PATH_TO_PACKAGES/%AG_WEBSERVICE_PKG/config/doctrine/schema.yml - has a high level description of the new table sfGuardUserProfile which contains some extras information about the users (web service clients). It's a easier way to manage who can access data;
  • ii) %PATH_TO_PACKAGES/%AG_WEBSERVICE_PKG/data/fixtures/fixtures.yml - inserts some active and inactive clients in database during web service installation/setup, as samples; it is worth to highlight that this approach is just a way to facilitate the development. The management of those clients will be done through a web interface (yet to be constructed);
  • iii) PATH_TO_PACKAGES/%AG_WEBSERVICE_PKG/lib/agWebservicesHelper.class.php - agWebservicesHelper - utility class whose functions are build and execute SQL/DQL queries and manipulate the results returning them in a appropriate format to the controller that is responsible for handling the request. It has static methods for each data point to be available in the web service;
  • iv) %PATH_TO_PACKAGES/%AG_WEBSERVICE_PKG/modules/agWebservices/actions/actions.class.php - agWebservicesActions - implements the necessary methods to the client's actions - defined in the routes file (%PATH_TO_PACKAGES/%AG_WEBSERVICE_PKG/modules/agWebservices/config/routing.yml);
  • v) %ROOT/test/functional/frontend/WebservicesActionsTest.php - test file (it tests routes, tokens and formats);

%ROOT = Path to project root directory.
%PATH_TO_PACKAGES = %ROOT/apps/frontend/lib/packages.
%AG_WEBSERVICE_PKG = name of directory where is the web service files, currently is "agWebservicesPackage".

Setup

First step (after install the Agasti) to activate the web service is check some permissions: Symfony framework has some directives to activate/deactivate plugins/packages, therefore, in %ROOT/config/ProjectConfiguration.class.php verify if there is the %AG_WEBSERVICE_PKG value in the array of enabled packages; Then, verify the existence of sf_guard_user_profile table in the database - otherwise, em %ROOT, execute the commands:

  echo "Generates table"
  ./symfony doctrine:create-model-tables sfGuardUserProfile
  echo "Generates models"
  ./symfony build-model                                      
  echo "Inserts sample data - delete the parameter to insert all data sample (declared in the fixtures.yml files)"
  ./symfony doctrine:data-load apps/frontend/lib/packages/agWebservicesPackage/data/fixtures/fixtures.yml

After that, PluginsfGuardUserProfile.class.php and PluginsfGuardUserProfileTable.class.php files should be generated in %PATH_TO_PACKAGES/%AG_WEBSERVICE_PKG/lib/model/doctrine.

A good way to check if the web service is ready is running the tests with the command:

  php %ROOT/test/functional/frontend/WebservicesActionsTest.php

IMPORTANT: before continue with any update, certify that you've committed the last modifications and have a database backup (the commands doctrine:build generate files and tables, therefore, modifications and data might be - and probably will be - lost. This may require to a new installation.

Security

In order to prevent data exposure, it was created a simple verification where only clients with a valid security key can request data. Hence, in the sfGuardUserProfile (%ROOT/lib/model/doctrine/agWebservicesPackage/sfGuardUserProfile.class.php file), we override the save method of sfGuardUserProfile model, so it can generate randomly the token for each client. This token is indispensable to request web service data. In the agWebservicesActions controller, we have

  $this->getRoute()->getObjects();

which executes the getByToken method of sfGuardUserProfileTable class (%ROOT/lib/model/doctrine/agWebservicesPackage/sfGuardUserProfileTable.class.php file) - it verifies whether it is a valid user (the route file explains why this).

Data points and formats

Currently, data can be downloaded in two patterns: JSON or XML. To choose between formats, simply type it in the URI. It is worth to remember also that it has only two reachable data points right now - organizations and staff.

URI & Routes

The web service routes are defined in %PATH_TO_PACKAGES/%AG_WEBSERVICE_PKG/config/routing.yml. In this file, it's defined the treatment for requests and the accepted URL patterns. Let's see the particular case “webservices_get”:

  • url: defines the accepted pattern; here the token, the data point and the format are parametrized
  • requirements: ensures the accepted values for the above parameters
  • class e options: adjusts the symfony framework to work with a type of route and which method of which class will be available in [code1]
  • param: defines an action of a controller which will handle the request

Available actions

  • List - /webservices/list/:datapoint - organizations or staff
  • Get - /webservices/:datapoint.sf_format/:token/:url_param - see URI & Routes and Filters
  • Index - /webservices/index OR /webservices - index action - it lists available actions

Filters

Now it's possible to filter data (only for staff data) based on parameters passed inside the URI. The :url_param should be a http query string (like a=1&b=2&c=3&d=4). There are predefined keys and values. Here's a simple documentation (possible keys/values):

  • is_available: 0|1 - whether staff is available
  • person: \d+ - which person id to filter
  • org: \d+ - which organization id to filter
  • limit: \d+ - how many staff to get
  • order: person|staff|organization - which order to use

For instance, if I want to grab 15 available staff ordered by organization, I should send a GET HTTP request to: /webservices/staff.json/my_token/is_available=1&limit=15&order=organization

Building a new web service or extending the current one

To create a new web service, the current structure can work as a bootstrap for new packages, with the appropriate modifications in case of a new approach. In this case, the *Setup* must be enough to get something working and to code the first steps. To extend the current web service, either to access new data points or to implement new features, it takes few adjusts:

  • i) write new tests that handle whatever is new;
  • ii) add new methods in the class, one for each new data point to be reachable;
  • iii) increment the method executeList in case of new functionalities;
  • iv) update the routing.yml if necessary (new actions);

Ideas for next implementation

There's a route for POST information. The idea is to send some JSON document inside the HTTP package and once received a well formated with proper values, Agasti could be able to insert this data into the database.

Project information

Important Links


QR Code
QR Code community:gsoc:gsoc_albuquerque (generated for current page)