Sfoglia categorie
Esplora
Fiverr Pro
Italiano
$
USD
I will build scalable etl pipelines using spark, databricks and delta lake
SDE1
I am a skilled Python developer with a strong focus on problem-solving, automation, and building robust data pipelines for big data analytics. My expertise lies in transforming data into actionable in...
Informazioni su questo servizio
Professional Data Engineering | ETL | Databricks | Spark | Data Lake Specialist
Are you looking for a scalable, production-ready data pipeline for your business?
I am a Professional Data Engineer with real industry experience building Big Data solutions using:
- Apache Spark / PySpark
- Databricks (Delta Lake, Medallion Architecture)
- Azure Data Factory
- Hadoop Ecosystem
- SQL & Python
- API Integrations (SOAP/REST)
- MongoDB, Oracle, MySQL, PostgreSQL, etc.
What I can help you with:
- ETL / ELT pipeline development
- Batch & real-time data processing
- Database Data Lake integrations
- Delta Lake architecture design
- Data cleaning & transformation
- Performance optimization
- Spark job tuning
- Automation & scheduling
- API Database ingestion
- Cloud data engineering (Azure/Databricks)
Real Experience Includes:
- MongoDB Delta Lake integration
- Oracle Fusion SOAP API Delta Lake ingestion
- Enterprise-level daily pipelines
- Optimized big data processing for millions of records
Why choose me?
⭐ Industry experience
⭐ Clean & optimized code
⭐ Scalable architecture
⭐ On-time delivery
⭐ 100% client satisfaction
FAQ
1. What services do you provide in this gig?
I provide professional Data Engineering and ETL/ELT services including: PySpark / Spark development Databricks workflows Azure Data Factory pipelines Database & API integrations Data Lake / Delta Lake setup Data cleaning & transformation Pipeline optimization & automation
2. Which technologies and tools do you work with?
I specialize in: Apache Spark / PySpark Databricks (Delta Lake, Notebooks, Jobs) Azure Data Factory Hadoop ecosystem Python & SQL REST/SOAP APIs MongoDB, Oracle, MySQL, PostgreSQL, SQL Server Cloud storage (ADLS, S3, Blob)
3. Can you integrate data from multiple sources?
Yes. I can integrate data from: Databases APIs CSV/Excel/JSON files Cloud storage Streaming sources
4. Do you build both ETL and ELT pipelines?
Yes. I support: ETL (Transform before load) ELT (Load first, transform in Databricks/Spark) Depending on your use case, I will recommend the best architecture for performance and cost.
5. Can you optimize my slow Spark or Databricks jobs?
Absolutely. I help with: Partition tuning Caching strategies Memory optimization Shuffle reduction Query optimization Delta Lake performance tuning This can significantly reduce execution time and cloud costs.
7. What do you need from me before starting?
Please provide: Source details (DB/API/files) Sample dataset or schema Expected output format Cloud platform details Business requirements Deadline Clear requirements help me deliver faster and better.
