S*****T

 Migrated data from Salesforce API to Lakehouse architecture for data product consumers.  Designed and implemented DLT (Delta Live Tables) pipelines to extract data from Salesforce and store it in Azure Data Lake Storage (ADLS) using Lake Flow Connect.  Built configuration files for DLT Meta pipelines to enable automated ingestion of multiple datasets.  Implemented Medallion Architecture (Bronze and Silver layers) for structured data transformation and quality improvements.  Created and managed Unity Catalog for centralized governance and data access control.  Integrated ingested data from ADLS into Unity Catalog for secure and organized consumption.  Performed data analysis and validation, ensuring accuracy before promoting datasets to DEV environment, followed by UAT environment for testing and sign-off.  Collaborated with stakeholders to handle schema drift and ensure smooth data ingestion and transformation processes.  Delivered scalable, reliable pipelines supporting downstream analytics and reporting. Technologies used: Azure Data Factory, Databricks, ADLS, Azure Monitor, Log Analytics, PySpark.

Lead Data and Integration:  Ld&I is a big data solution to integrate data from APIs, perform transformations in Azure Databricks (ADB), and orchestrate data pipelines in Azure Data Factory (ADF). Developed and managed Azure Function Apps to execute batch processes efficiently.  Worked on reducing the response time of the dedicated SQL Pool in Azure synapse workspace and deployed the changes across all environments using Azure Devops(ADO) release pipelines.  Collaborating with the DevOps team to test the migration from classic to YAML pipelines.  Addressed ICM incidents related to pipeline failures and resolved S360 items. Technologies used: Azure: Synapse, Databricks, ADF, Logic App, AppInsights, Function App. GPS:  GPS is an End-to-End BI platform from data acquisition to reporting. The platform is used to generate reports, dashboards & scorecards related to various aspects of Microsoft Consulting & Premier services.  It also provides processed data to the downstream system for consumption. Through ETL Pipelines the data flows into the reports.  Working exposure to Agile methodology.  Fixing the data discrepancy in reports and analyzing the code in Databricks from our end. Technologies used: Databricks, ADF, PowerBI.