Urban Observatory Sensor - Ismena website

info@ismena.com

Urban Observatory Sensor Connector

Connector Details

Type

Virtual machines, Single VM , BYOL

Runs on

Google Compute Engine

Last Update

24 October, 2024

Category

Overview

The Urban Observatory Sensor Connector enables seamless integration with the Urban Observatory Sensor API, providing access to sensor locations, sensor data, variable metadata, and theme metadata. This connector acts as a proxy to streamline data retrieval, supporting actions for listing sensor locations, downloading sensor data in various formats, retrieving pagination information, and accessing metadata about available variables and themes. It supports JSON, CSV, and ESRI Shapefile (.zip) response formats and is designed for use with Google Cloud Platform (GCP) environments.

Integration Overview

This document provides a detailed guide for each integration point, its purpose, configuration, and workflow support using the Urban Observatory Sensor Connector within a GCP environment.

listSensorsJson: Retrieves a list of sensor locations in JSON format, with optional bounding box filtering and pagination.
getSensorsPage: Retrieves pagination metadata for sensor datasets, including total count and navigation URLs.
downloadSensorsCsv: Downloads sensor locations as a CSV file, with optional filtering and pagination.
downloadSensorsShp: Downloads sensor locations as a zipped ESRI Shapefile, with optional filtering and pagination.
getSensorsDataJson: Retrieves sensor data in JSON format, with optional time range, variable, and bounding box filters.
getSensorsDataCsv: Downloads sensor data as a CSV file, with optional filters.
getSensorIndDataJson: Retrieves data for a specific sensor in JSON format, with optional filters.
getSensorIndDataCsv: Downloads data for a specific sensor as a CSV file, with optional filters.
listVariables: Retrieves a list of available sensor measurement types and their metadata.
listThemes: Retrieves a list of available themes for sensor variables.

Detailed Integration Documentation

List Sensors JSON Retrieval

Action	listSensorsJson
Purpose	Retrieves a comprehensive list of sensor locations in JSON format, optionally filtered by bounding box coordinates. Useful for geospatial applications or dashboards.
Parameters	Required: None Optional: limit (integer, default: 1000, -1 to disable pagination) offset (integer, default: 0) bbox_p1_x, bbox_p1_y (lower-left longitude & latitude) bbox_p2_x, bbox_p2_y (upper-right longitude & latitude)
Configuration	Set `CONNECTOR_ENV_URBANOBSERVATORY_BASE_URL` in GCP (e.g., https://api.v2.urbanobservatory.ac.uk).
Output	Successful: JSON with sensor details (name, location, centroid coordinates, height, broker, ID) and pagination metadata. Failure: JSON validation errors (HTTP 422).
Workflow Example	Deploy connector to GCP → Call listSensorsJson with limit=10 → Use JSON response to populate a geospatial visualization (e.g., Google Maps API).

Get Sensors Page Retrieval

Action	getSensorsPage
Purpose	Retrieves pagination metadata for sensor datasets, including total count and page navigation links. Useful for managing large datasets.
Parameters	Required: None Optional: limit (integer, default: 1000) offset (integer, default: 0) bbox_p1_x, bbox_p1_y (lower-left coordinates) bbox_p2_x, bbox_p2_y (upper-right coordinates)
Configuration	Set `CONNECTOR_ENV_URBANOBSERVATORY_BASE_URL` in GCP.
Output	Successful: JSON with total sensors, limit, offset, next, and previous page URLs. Failure: JSON validation errors (HTTP 422).
Workflow Example	Deploy on GCP Cloud Run → Call getSensorsPage with limit=100 → Use metadata (Total, Next) to implement paginated data retrieval in pipelines.

Download Sensors CSV

Action	downloadSensorsCsv
Purpose	Downloads sensor locations as a CSV file, including core metadata like sensor name, location, and height. Supports bounding box filtering and pagination.
Parameters	Required: None Optional: limit (integer, default: 1000, -1 to disable pagination) offset (integer, default: 0) bbox_p1_x, bbox_p1_y (lower-left longitude & latitude) bbox_p2_x, bbox_p2_y (upper-right longitude & latitude)
Configuration	Set CONNECTOR_ENV_URBANOBSERVATORY_BASE_URL in GCP. Use header: Accept: text/csv.
Output	Successful: CSV file with sensor_name, Location_WKT, centroid coordinates, heights, broker name, and raw ID. Failure: JSON error object (HTTP 422).
Workflow Example	Configure GCP connector → Call downloadSensorsCsv with limit=100 and Accept: text/csv → Store CSV in Google Cloud Storage for BigQuery processing.

Download Sensors Shapefile

Action	downloadSensorsShp
Purpose	Downloads sensor locations as a zipped ESRI-compatible Shapefile (.shp, .shx, .dbf, .prj, .cpg). Ideal for GIS applications.
Parameters	Required: None Optional: limit (integer, default: 1000, -1 to disable pagination) offset (integer, default: 0) bbox_p1_x, bbox_p1_y (lower-left longitude & latitude) bbox_p2_x, bbox_p2_y (upper-right longitude & latitude)
Configuration	Set CONNECTOR_ENV_URBANOBSERVATORY_BASE_URL in GCP. Use header: Accept: application/zip.
Output	Successful: Zipped ESRI Shapefile with sensor location data. Failure: JSON error object (HTTP 422).
Workflow Example	Deploy connector on GCP → Call downloadSensorsShp with limit=100 and Accept: application/zip → Store Shapefile in Google Cloud Storage and integrate with Google Earth Engine.

Get Sensors Data JSON Retrieval

Action	getSensorsDataJson
Purpose	Retrieves sensor data in JSON format with optional filters for time range, variables, and bounding box. Useful for real-time sensor data analysis.
Parameters	Required: None Optional: start, end (UTC datetime strings or null) last_n_hours (integer > 0, overrides time range) last_n_days (integer > 0, overrides time range) variables (array of strings or null, e.g., ["O3","NO2"]) bbox_p1_x, bbox_p1_y (lower-left longitude & latitude) bbox_p2_x, bbox_p2_y (upper-right longitude & latitude) limit (integer, default: 1000, -1 to disable pagination) offset (integer, default: 0)
Configuration	Set base URL in GCP. Use header: Accept: application/json.
Output	Successful: JSON object with: Readings (array: Sensor_Name, Variable, Value, Timestamp, Flagged) Sensors (metadata, keyed by Sensor_Name) Start, End (query timestamps) Pagination metadata Failure: JSON error object (HTTP 422).
Workflow Example	Deploy connector on GCP Cloud Run → Call getSensorsDataJson with limit=10 and last_n_hours=24 → Process response in GCP Pub/Sub pipeline for real-time analytics.

Get Sensors Data CSV Retrieval

Action	getSensorsDataCsv
Purpose	Downloads sensor data as a CSV file with optional filters for time range, variables, and bounding box. Ideal for bulk data export.
Parameters	Required: None Optional: start, end (UTC datetime strings or null) last_n_hours (integer > 0, overrides time range) last_n_days (integer > 0, overrides time range) variables (array of strings or null) bbox_p1_x, bbox_p1_y (lower-left longitude & latitude) bbox_p2_x, bbox_p2_y (upper-right longitude & latitude) limit (integer, default: 1000, -1 to disable pagination) offset (integer, default: 0)
Configuration	Set base URL in GCP. Use header: Accept: text/csv. Start with small limits (e.g., 10) to avoid server 500 errors.
Output	Successful: CSV file with: sensor_name, variable, value, timestamp, flagged Location_WKT, centroid coordinates, heights, broker name, raw ID Failure: JSON error object (HTTP 422).
Workflow Example	Configure connector in GCP → Call getSensorsDataCsv with limit=10, last_n_hours=24, Accept: text/csv → Store CSV in Google Cloud Storage and analyze with BigQuery.

Get Sensor Individual Data JSON Retrieval

Action	getSensorIndDataJson
Purpose	Retrieves data for a specific sensor in JSON format with optional filters for time range and variables. Useful for detailed sensor analysis.
Parameters	Required: sensor_name (string, e.g., "PER_AIRMON_MESH1916150") Optional: start, end (UTC datetime strings or null) last_n_hours (integer > 0) last_n_days (integer > 0) variables (array of strings or null) limit (integer, default: 1000, -1 disables pagination) offset (integer, default: 0)
Configuration	Set base URL in GCP. Use header: Accept: application/json.
Output	Successful: JSON object with same structure as getSensorsDataJson. Failure: JSON error object (HTTP 422).
Workflow Example	Deploy connector on GCP Cloud Functions → Call getSensorIndDataJson with sensor_name=PER_AIRMON_MESH1916150 and last_n_hours=24 → Use response in dashboard for sensor-specific monitoring.

Get Sensor Individual Data CSV Retrieval

Action	getSensorIndDataCsv
Purpose	Downloads data for a specific sensor as a CSV file with optional filters. Ideal for exporting sensor-specific data for analysis.
Parameters	Required: sensor_name (string) Optional: start, end (UTC datetime strings or null) last_n_hours (integer > 0) last_n_days (integer > 0) variables (array of strings or null) limit (integer, default: 1000, -1 disables pagination) offset (integer, default: 0)
Configuration	Set base URL in GCP. Use header: Accept: text/csv. Start with small limit values to avoid 500 errors.
Output	Successful: CSV file with same structure as getSensorsDataCsv. Failure: JSON error object (HTTP 422).
Workflow Example	Configure connector in GCP → Call getSensorIndDataCsv with sensor_name=PER_AIRMON_MESH1916150, limit=10, last_n_hours=24 → Store CSV in Google Cloud Storage for analysis with Dataflow.

List Variables Retrieval

Action	listVariables
Purpose	Retrieves a list of available sensor measurement types, including their units, limits, and themes. Ideal for populating UI filter menus.
Parameters	Required: None Optional: None
Configuration	Set base URL in GCP. Use header: Accept: application/json.
Output	Successful: JSON object with: Variables: Array of objects containing Name, Units, Upper_Limit, Lower_Limit, Theme Failure: JSON error object (HTTP 422)
Workflow Example	Deploy connector on GCP Cloud Run → Call listVariables → Use response to populate variable selection dropdown in web app.

List Themes Retrieval

Action	listThemes
Purpose	Retrieves a list of available themes for sensor variables, useful for categorizing data in applications.
Parameters	Required: None Optional: None
Configuration	Set base URL in GCP. Use header: Accept: application/json.
Output	Successful: JSON object with: Themes: Array of objects containing Name Failure: JSON error object (HTTP 422)
Workflow Example	Deploy connector on GCP Cloud Functions → Call listThemes → Use response to categorize sensor data in BigQuery dashboard.

Workflow Creation with the Connector

Example Workflow: Sensor Data Collection and Analysis

Retrieve Sensor Locations	Use the listSensorsJson action with limit=10 and optional bounding box parameters (e.g., bbox_p1_x=-1.6, bbox_p1_y=55.0, bbox_p2_x=-1.5, bbox_p2_y=55.1) to fetch sensor locations. Store the JSON response in Google Cloud Storage.
Query Sensor Data	Execute the getSensorsDataCsv action with limit=10, last_n_hours=24, and variables=["O3"] to retrieve recent air quality data. Save the CSV to Google Cloud Storage and load it into BigQuery for analysis.
Analyze and Visualize	Use the listVariables and listThemes actions to fetch metadata for UI filters. Deploy a GCP-hosted web application using App Engine to visualize sensor data with Google Maps and Data Studio, leveraging the collected data and metadata.

This workflow enables applications to collect, store, and analyze sensor data within GCP, supporting real-time monitoring and geospatial analysis.

Pricing

bill-and-calculator-2023-11-27-05-25-06-utc 1

Request a Quote

Support

For Technical support please contact us on

custom-connectors-support@isolutions.sa

A tech solution company dedicated to providing innovation thus empowering businesses to thrive in the digital age.

Register to IBM x iSolution Event