Case study 04
Industrial IoT data pipeline
Transporting irregular device telemetry from Raspberry Pi data loggers into a remotely queryable cloud store.
Industrial solar loggers emitted large form-encoded payloads containing CSV headers and readings rather than application-ready JSON. The data needed to cross a public API boundary, be transformed safely, and become useful for remote monitoring.
This was an early applied cloud project: the goal was a working ingestion path, with security and production hardening still explicitly incomplete in the repository documentation.
Each choice connects a business constraint to an implementation boundary and retained evidence.
Used a Velocity Template Language mapping to convert application/x-www-form-urlencoded input into the event shape expected by Lambda.
Evidence · The README documents the original payload format, mapping boundary, headers, status codes, and API flow.Parsed the first CSV row as headers and zipped later values into a nested values object keyed by the original telemetry names.
Evidence · The repository's Lambda example shows header extraction and MongoDB document construction.Provided PyMongo through a Lambda layer and kept database connectivity inside the short-lived function execution path.
Evidence · The repository explains the external library bundle and MongoDB connection lifecycle.The pipeline accepted device readings through API Gateway, transformed them, inserted structured timestamped records into MongoDB, and made remote generation data available for visualization work.
The case study also preserves the honest limitation: endpoint secrecy was not a security model, and input validation, authentication, and rate controls needed a proper production pass.
Evidence basis