Problem

DeepEcho needed an architecture for their DataOps side, including an ingestion system and preprocessing operation. They also required a front-end web application for ultrasound video ingestion and an anonymization tool. The Sparks Foundation needed machine learning models developed and trained on various tasks, while Direction Générale de la Météorologie (DGM) required a simple and efficient react-native weather forecasting application.

Solution

We designed and implemented a scalable Snowflake database architecture for DeepEcho's efficient data storage and retrieval, incorporating data vault modeling. We also developed automated data transformation workflows using dbt for streamlined processing and integrated dbt vault for secure data handling. For the front-end web application, we used Airflow, PostgreSQL, Azure Cloud Service, Flask, Docker FastAPI, RestAPI, Rclone. For the anonymization tool, we utilized similar tools such as Flask and Docker.

For The Sparks Foundation's machine learning needs, we utilized Linear Regression, Logistic Regression, Decision Tree, K-Means SVM to develop and train models on various tasks. Finally, for DGM's weather forecasting app development needs we built an exclusively react-native application that provides current weather forecasts as well as forecasts every day over 5 days.

Impact

Efficient data storage and retrieval with scalable Snowflake database architecture
Streamlined processing with automated data transformation workflows using dbt
Precise weather forecasting with DGM's exclusively react-native application

Team

Team Member #1 - DataOps Engineer Intern
- Designed and implemented scalable Snowflake database architecture for efficient data storage and retrieval, incorporating data vault modeling
- Developed automated data transformation workflows using dbt for streamlined processing and integrated dbt vault for secure data handling
Team Member #2 - Mobile App Developer
- Built an exclusively react-native application that provides current weather forecasts as well as forecasts every day over 5 days

Scalable DataOps & ML Solutions with Snowflake & React-Native