Setting up near-real time data pipelines in Azure, with Azure Stream Analytics allows one to quickly scale and perform advanced analytics on moving data. The following repository demonstrates how we easily utilize the following Azure components to easily spin up a data pipeline to stream data from multiple sources
- Azure WebJob
- Azure EventHub
- Azre Stream Analytics
- Azure Functions
- Azure DevOps
- Power BI
- Raspberry PI
- Twitter Streaming API
- GrovePi
- Twilio
This repo demonstrates the ability to stream Twitter data based on keywords or specific users utilizing Azure WebJobs, EventHub and Stream Analytics. The egressed and analysed data can be vizualized in a Power BI dashboard, or acted upon based on triggered Azure Functions that will send e-mail notifications through a Sendgrid API. The data is furthermore stored in a CosmosDB for later use.
This repo demonstrates the ability to stream sensor data from a Raspberry PI utilizing Azure EventHub and Stream Analytics. The sensor data can be viewed in a Power BI dashboard, but there is also built in functionality to demonstrate how easy it is to set up your on burglar alarm. Any motion detected by the Raspberry PI's sensors will trigger an Azure Function which will call the user using Twilio. The sample demonstrates the LAG functionality in particular, but also how to use reference data to enrich a stream.
Please follow the following guide to setup and run semantic analysis
Please follow the following guide to setup and run Raspberry PI sensor streaming
See anything you want to improve? Do you want to build out the existing code base? Don't heistate to open a PR! Please read our contributing guidelines