Python - Data Test

Focuses on native Python data structures like lists, dictionaries, tuples, sets, stacks, and queues. Covers comprehension patterns, hashing, sorting/searching algorithms, recursion, and algorithmic thinking. Includes use of time/space complexity to evaluate code efficiency, especially when dealing with data-intensive loops or transformations. Essential for writing performant data manipulation logic.

Evaluates fluency in manipulating tabular data using pandas: indexing, slicing, merging, grouping, pivoting, reshaping, handling time-series data, and chained operations. Also includes performance optimizations using vectorized ops, categoricals, and memory profiling. Tests candidate’s ability to transform messy or multi-source datasets into structured, analysis-ready formats.

Covers use of NumPy for numerical data: creating/mutating arrays, broadcasting rules, matrix operations, statistical aggregations, memory layout, and efficiency considerations in large-scale array processing. Also includes integration with pandas and use of ufuncs and structured arrays. Enables data professionals to write highly optimized, vectorized code instead of iterative loops.

Tests ability to handle real-world, messy datasets: null values, outliers, inconsistent formats, mixed data types. Includes use of regex, str methods, parsing datetime, normalization/scaling, and encoding techniques (label, one-hot). Also evaluates logic behind conditional transformations and feature construction in preparation for ML or reporting workflows.

Assesses statistical and visual intuition for identifying patterns, trends, anomalies, and relationships. Includes descriptive stats, histogram/skewness analysis, correlation heatmaps, box/violin plots, time-series decomposition, and interactive visualizations using matplotlib, seaborn, and plotly. Tests ability to tell a compelling data story through visuals and derive hypotheses.

Evaluates candidate’s ability to retrieve, process, and persist data across multiple sources/formats. Includes file operations (open(), with, CSV, JSON, Excel), REST API interaction via requests, parsing nested JSON/XML, and integrating with databases via SQLAlchemy. Tests practical web integration and automation skills required for modern data ingestion workflows.

Focuses on scalable data processing using PySpark: RDD vs DataFrame APIs, transformations/actions, joins, schema inference, lazy evaluation, partitioning, caching, and performance tuning. Also includes reading/writing from HDFS/Parquet, and integrating with SQL and structured streaming. Essential for handling large volumes of data in production data pipelines.

Assesses readiness to apply ML workflows using scikit-learn: preprocessing pipelines, feature selection, training classification/regression models, evaluating performance using accuracy, AUC, RMSE, precision/recall. Also includes hyperparameter tuning (e.g., GridSearchCV), model validation, and cross-validation strategies. Evaluates applied understanding of ML best practices and analytical modeling.

Tests advanced competencies in building production-grade data systems. Includes orchestration with Apache Airflow, API deployment using Flask/FastAPI, containerization via Docker, and CI/CD concepts. Covers cloud data services (S3, GCS, Azure Blob), secrets/config handling, ML model serving, and tracking with MLflow. Also touches on governance, access control, and pipeline resilience.