Campbelltown Open Data Connector

info@ismena.com

Connector Details

Type

Virtual machines, Single VM , BYOL

Runs on

Google Compute Engine

Last Update

24 October, 2024

Category

The Campbelltown Open Data Connector provides seamless integration with the City of Campbelltown’s public open data portal, enabling secure and efficient access to all published datasets, records, metadata, and file exports. This connector acts as a dedicated integration layer to streamline data retrieval, supporting actions for dataset discovery, record querying, full exports in multiple formats, and attachment access.

Integration Overview

This document provides a detailed guide for each integration point, its purpose, configuration, and workflow support using the Campbelltown Open Data Connector.

Supported Integration Action Points

listDatasets: Retrieves a list of all available datasets.
getDatasetMetadata: Retrieves detailed metadata for a specific dataset.
queryRecords: Queries records from a dataset with full filtering and pagination support.
getSingleRecord: Retrieves a specific record by its unique ID.
listExportFormats: Lists all available export formats for a dataset.
exportDataset: Exports an entire dataset in the requested format (csv, json, geojson, kml, shp, parquet, xlsx, etc.).
listAttachments: Retrieves file attachments associated with a dataset.

Detailed Integration Documentation

2.1 List All Datasets

Action	GET /catalog/datasets
Purpose	Retrieves a comprehensive list of all publicly available datasets on the Campbelltown Open Data portal. Serves as the primary entry point for discovering available data.
Configuration	Ensure the connector is configured with the base URL via the CONNECTOR_ENV_CAMPBELL_BASE_URL environment variable.
Parameters	Required: None Optional: Standard query parameters (e.g., limit, offset, q, facet)
Output	Successful: JSON object containing dataset list with metadata (title, description, dataset_id, etc.) Failure: Returns upstream error details (e.g., service unavailable)
Workflow Example	Execute the listDatasets action Process the response to populate a data catalog or discovery interface

2.2 Get Dataset Metadata

Action	GET /catalog/datasets/<dataset_id>
Purpose	Retrieves complete metadata for a specific dataset, including fields, descriptions, publisher, and update frequency.
Configuration	Same base URL configuration as for listing datasets.
Parameters	Required: dataset_id (path parameter) Optional: None
Output	Successful: Returns full dataset metadata object in JSON Failure: Returns error details (e.g., dataset not found)
Workflow Example	Execute the getDatasetMetadata action with a known dataset_id Use field definitions to generate dynamic forms or data tables

2.3 Query Dataset Records

Action	GET /catalog/datasets/<dataset_id>/records
Purpose	Performs powerful queries against a dataset with support for filtering, sorting, pagination, and field selection.
Configuration	Same base URL configuration as above.
Parameters	Required: dataset_id (path parameter) Optional: Query parameters such as where, refine, select, order_by, limit, offset
Output	Successful: Returns JSON with total_count, records array, and pagination information Failure: Returns validation or upstream error details
Workflow Example	Execute the queryRecords action with filters (e.g., where=status="Approved") Display results in dashboards, maps, or reporting tools

2.4 Get Single Record

Action	GET /catalog/datasets/<dataset_id>/records/<record_id>
Purpose	Retrieves a single record using its unique identifier.
Configuration	Same base URL configuration as above.
Parameters	Required: dataset_id, record_id (path parameters) Optional: None
Output	Successful: Returns complete record object Failure: Returns error (e.g., record not found)

2.5 List Available Export Formats

Action	GET /catalog/datasets/<dataset_id>/export-formats
Purpose	Returns all supported export formats for a given dataset.
Configuration	Same base URL configuration as above.
Parameters	Required: dataset_id (path parameter) Optional: None
Output	Successful: Returns JSON array of format identifiers (e.g., csv, json, geojson, shp, parquet) Failure: Returns error details

2.6 Export Dataset

Action	GET /catalog/datasets/<dataset_id>/exports/<format>
Purpose	Streams a complete export of the dataset in the specified format. Ideal for bulk data analysis, backups, or integration with BI tools.
Configuration	The connector uses streaming responses and extended timeouts to efficiently handle large file exports.
Parameters	Required: dataset_id, format (path parameters) Optional: Export-specific parameters (e.g., delimiter, timezone)
Output	Successful: Streams file with correct Content-Type and Content-Disposition headers Failure: Returns upstream error or timeout
Workflow Example	Execute the exportDataset action with desired format (e.g., parquet) Pipe the stream directly into storage or analytics platforms

2.7 List Dataset Attachments

Action	GET /catalog/datasets/<dataset_id>/attachments
Purpose	Retrieves metadata and download links for files attached to a dataset (e.g., PDFs, supporting documents).
Configuration	Same base URL configuration as above.
Parameters	Required: dataset_id (path parameter) Optional: None
Output	Successful: Returns JSON array of attachment objects (title, URL, mime type) Failure: Returns error details
Workflow Example	Execute the listAttachments action Display document links alongside dataset records in applications

Example Workflow

Example Workflow: Building a Community Data Portal

Example Workflow: Building a Community Data Portal

Retrieve Supported Datasets	Use the listDatasets action to fetch all available datasets Populate a searchable catalog for end users
Explore Dataset Details	Execute getDatasetMetadata for selected datasets to understand structure and fields
Power Interactive Applications	Use queryRecords with dynamic filters to drive maps, charts, and tables
Enable Bulk Data Access	Trigger exportDataset in formats like CSV or Parquet for researchers and analysts
Enrich User Experience	Use listAttachments to provide direct access to supporting documents and reports

This connector enables robust, scalable, and performant integration with Campbelltown City Council’s open data platform, supporting everything from real-time dashboards to large-scale data extraction and analysis.

Pricing

bill-and-calculator-2023-11-27-05-25-06-utc 1

Request a Quote

Support

For Technical support please contact us on

custom-connectors-support@isolutions.sa

A tech solution company dedicated to providing innovation thus empowering businesses to thrive in the digital age.

Register to Wired for Banking