Introduction
Databricks is a powerful platform for huge data analytics, machine learning, and ETL pipelines. It’s optimized for processing massive datasets, now not coping with transactional workloads.
However, many developers wonder:
“Can I use Databricks at once in my Angular + NestJS app to keep and control statistics?”
The answer: No — and here’s why.
In this post, we’ll break down why Databricks need to no longer needs to be used for direct information storage in regular internet packages, and why PostgreSQL (or different relational/NoSQL databases) are the right fit for dealing with your app’s data.
Databricks isn’t always built for Transactional Workloads
Databricks runs on Apache Spark, which is designed for batch processing and analytical workloads — not real-time information storage.
Why these subjects:
- No low-latency INSERTs/UPDATEs for real-time records
- Lacks ACID transaction guarantees at a granular level
- Not optimized for row-level writes
- Not ideal for concurrent user transactions
- This makes Databricks unfit for real-time applications like:
- User sign-ups
- Orders & payments
- Form submissions
- Admin panels
- CRUD operations
You Can’t Use Databricks from Angular Directly
Just like in our earlier blog, you cannot securely call Databricks directly from Angular:
- Databricks APIs require authentication via Bearer tokens
- Tokens can’t be exposed in frontend apps
- Databricks APIs don’t support CORS (Cross-Origin Resource Sharing)
Example failure in Angular:
Will be blocked due to CORS + token exposure
Even NestJS Shouldn’t Write Directly to Databricks
Databricks is not a replacement for a transactional database — even if you try to write data using a backend like NestJS.
Challenges:
- Writes are expensive and slow
- No real-time indexing for fast reads
- Managing data integrity is difficult
- You’ll often be writing to Delta Lake files, not traditional tables
Use Databricks for:
- Batch ingesting log data
- Analyzing large time-series datasets
- Running ML pipelines
- Generating reports
Use PostgreSQL or Other Databases for Application Data
For reliable, real-time data handling in web applications, you need a database designed for transactions.
Recommended Options:
- PostgreSQL – Open-source, powerful, widely adopted
- MySQL – Great for read-heavy applications
- MongoDB – NoSQL, flexible schema
- Firebase / Supabase – For real-time apps
How it fits:
- Angular ⟶ makes HTTP calls to ⟶ the NestJS API
- NestJS ⟶ handles logic, validation, and writes data to ⟶ PostgreSQL
- Optional: Data gets synced periodically to Databricks for analytics
Ideal Architecture
Here’s a reliable and scalable setup:
Why This Works:
- You get low-latency, secure, real-time transactions with PostgreSQL
- You can perform heavy analytics on large volumes in Databricks without slowing down your app
- The roles are clearly separated:
- Database handles CRUD
- Databricks handles analytics
- NestJS handles logic
- Angular handles UI
How to Sync Data to Databricks
Use ETL (Extract, Transform, Load) tools like:
- Airbyte
- Fivetran
- Apache NiFi
- Custom Python scripts
These tools can push data from PostgreSQL to Databricks regularly or in real-time using streaming.
This allows you to:
- Store data in PostgreSQL
- Sync it to Databricks
- Run reports and visualizations on massive datasets
Conclusion
Databricks is not a transactional database. It’s a data lake and analytics engine — perfect for big data but not for storing user records, handling real-time updates, or replacing your application database.
For any real-world Angular + NestJS application:
- Store data in PostgreSQL or MongoDB
- Use NestJS as your backend
- Use Databricks for analytical processing only
Separate your responsibilities — that’s the key to building secure, scalable, and maintainable systems.