Data & Analytics Technologies We Work With
From cloud compute and data warehouses to pipelines, streaming, and machine learning, these are the tools behind the 21 projects in our portfolio. Python runs through every one of them, and each platform listed here has been used to deliver real work for real clients.
We start with the stack you already have and only suggest something new when it is clearly worth it. With no vendor affiliation and no referral fees, every recommendation is based on your data, your team, and your goals.
Layer by Layer Modern Data Stack
A modern data stack refers to a set of cloud services that move data from the collection point to the decision-making point, including collection, storage, processing, and visualisation options. We are using your existing stack while suggesting new technology only when it is worth it.
Technology We Use
Every mentioned technology is used in one of the projects specified in our portfolio.
“Used in X number of projects” means “the projects in the portfolio are numbered”. Companies are named as they are listed in our portfolio.
Cloud and Compute
Serverless jobs and containers that execute your pipelines without need to manage servers.
AWS
Lambda and ECS for pipeline jobs and DynamoDB for high-speed lookups.
Used in 5 delivered projects: Like Runwise, Pulley, Callbox and Sifted.
Data engineeringDocker
Services that can be reused multiple times and act the same in production and development.
Used in 5 delivered projects: Including Chili Piper, Agora, and Banyan Technology
Data engineeringAnalysis & Machine Learning
From the clean tables to forecasts and predictions.
Python
The main language we use to develop pipelines, analyze data, and create models.
Used in all of the 21 published projects: Every project from our portfolio
Predictive analyticsPandas & SQLAlchemy
Preparing and modeling data in code that is readable and testable.
Used in 6 published projects: Among them: Sifted, Pulley, FourKites, Census
Data analyticsWarehouses and Databases
Where data is kept and prepared for analysis.
Snowflake
Cloud-based warehouse for heavy-duty analytics and ingestion.
Used in 4 delivered projects: Datasembly, Census, 4Kites and Aera Technology
Data engineeringGoogle BigQuery
Serverless warehouse for event and marketing information.
Used in 3 delivered projects: Digible, Aera Technology and Root Insurance
PostgreSQL
The database providing reliable storage for most of the data applications we deal with.
Used in 7 delivered projects: Including Runwise, Sifted, Chili Piper and Agora
Data engineeringRedis
In-memory queues and caching for real-time features.
Used in 6 delivered projects: Including ShelterZoom, Sendoso and Attentive
Data engineeringPipelines, Streaming and Orchestration
Data transmission by schedule or in real time with retries and alerts.
Apache Airflow
Pipelines that can be scheduled and monitored instead of depending on erratic cron jobs.
Use of 3 published projects: Census, Aera Technology, Root Insurance
Data engineeringApache Kafka
Stream processing for events that need to take place without waiting for night’s batch processing.
Use of 2 published projects: Datasembly, Root Insurance
Scrapy & BeautifulSoup
Collecting data from the Web in such a way as to be sure that it won’t fail silently.
Use of 4 published projects: Among them: Digible, Callbox, Roofstock
Data miningFastAPI & Django REST
API to provide your data and models to other systems.
Use of 9 published projects: Among them Chili Piper, Innovaccer, Banyan Technology
Data engineeringAlso in Our Toolkit
In this category, we provide tools that our team is familiar with, even if we have not had the opportunity to work on a project with a client yet. Ask us during a call to find out the kind of tools we use.
How We Choose The Right Platform For You
Most projects start at a stack you already have.
Volume and speed
Nightly batch, hourly, or real-time streaming
Cost structure
Pricing for compute, storage and data transfer.
Who maintains it
Your team’s capabilities after we finish the process.
What you’re currently using
Licenses, clouds, and tools that your team is familiar with. (Button: Send us your stack)
Governance
It refers to safeguarding access, auditing, as well as locations for data storage.
We have no vendor affiliation and earn no referral fee from insurance companies, which means that our recommendations are purely based on your requirements, not on sales targets.
Technology Questions
Asked QuestionsQ1. Are you an official partner of Snowflake, Microsoft, or AWS?
No, we are independent consultants who use these platforms to offer you what works best for your data, instead of promoting what we have for sale.
Q2. What platforms have you used in your client projects?
In our 21 projects, we have worked with AWS, Docker, Snowflake, Google BigQuery, PostgreSQL, Redis, Apache Airflow, Apache Kafka, Scrapy, FastAPI and Django REST, and have used Python in all of them.
Q3. Do you utilize the tools we have available?
Definitely. All projects commence using the same technology you currently own, and we only suggest the adoption of a new technology when it becomes beneficial in the long run.
Q4. Do you create dashboards in Tableau and Power BI?
Yes. Both of these tools are part of our capabilities. Presently, no project case study has been published on this subject, so please ask for samples during a call.
Q5. On what basis do you choose between Snowflake and BigQuery?
We analyze and compare the variables of volume and speed, ascertain the type of support and maintenance, pricing strategy, and issues of governance. We have already accomplished several projects covering both of these technologies: four projects for Snowflake and three for BigQuery, which supports our being a leader in this field.