I will help you with apache spark machine learning pipelines

I'm a Pyspark developer, here to help you, or develop for you, your project. You can specify if you prefer source code, Jupyter notebook or Databricks notebook.

I can develop your ML Pipelines with Cross Validation and perform Hyperparameter tuning for every ML algorithm you like.

Data visualization can be done via Databricks or, when possible, with pandas, matplotlib, seaborn, plotly (interactive plots). In case of supervised learning we can make a ROC and PR analysis with the help of scikit-learn.

1. Pre Processing:

• dealing with missing data
• imputation
• indexing and encoding categorical variables
• standardization and scaling
• standalone or with pipelines

2. Machine Learning:

• Classification
• Regression
• Clustering
• Pipelines
• Cross Validation
• Hyperparameter tuning
• Collaborative filtering (ALS)

3. Data visualization:

• Databricks
• Matplotlib
• Seaborn
• Plotly