• Posts
AI
• Computer Vision Models
  • Segment Anything (SAM)
  • SEEM: Segment Everything Everywhere All at Once
• Data Engine
• Deep Learning Algorithms
• Dataset Lineage
Data Engineering
• Fast Data Lake
• Immutable Data
Software Engineering
• Tech Debt

From Chaos to Clarity: How Immutable Data Simplifies Machine Learning Data Storage

Karpathy’s data engine concept is a data management strategy designed to simplify the process of working with large and complex datasets in machine learning applications. Potentially nitpicky but competitive advantage in AI goes not so much to those with data but those with a data engine: iterated data aquisition, re-training, evaluation, deployment, telemetry. And whoever can spin it fastest. Slide from Tesla to ~illustrate but concept is general pic.twitter.com/6O2KxZBg17 — Andrej Karpathy (@karpathy) December 5, 2022 Developed by Andrej Karpathy, former Director of AI at Tesla, the data engine concept aims to streamline the process of data collection, preprocessing, and transformation, allowing machine learning practitioners to focus on the development and deployment of their models.

March 20, 2023 Read

Democratise your Data Lake with FastAPI and Cassandra

Motivation A data platform can be thought of as a scalable system for ingestion, curation, transformation and integration of datasets. But the value of a data platform is realised when these datasets are made available for use outside the platform to drive decisions and shape the customer experience, i.e. by democratising the data. Data Democratisation may come in many shapes and sizes, with efforts typically starting with a Data Warehouse exposed via a BI framework (e.

August 10, 2020 Read