Sr Databricks/Agentic AI

Hi, Greetings from DIA SOFTWARE SOLUTIONS LLC! We reaching out about an exciting Direct client opportunity with one of our clients. Please review the requirements and let me know if you are interested in this position? Direct client Req:: Need Sr Databricks/Agentic AI ,Remote, TX PLEASE SEND THE RESUMES TO SKUMAR AT DIASOFTWARESOLUTIONS DOT COM ! Job Description: The Worker is responsible for developing, maintaining, and optimizing big data solutions using the Databricks Unified Analytics Platform. This role supports data engineering, machine learning, and analytics initiatives within this organization that relies on large-scale data processing. Duties include: Designing and developing scalable data pipelines Implementing ETL/ELT workflows Optimizing Spark jobs Integrating with Azure Data Factory Automating deployments Collaborating with cross-functional teams Ensuring data quality, governance, and security Build AI accelerators and reusable components to boost development teams productivity. SKILLS MATRIX Minimum Requirements: Candidates that do not meet or exceed the minimum stated requirements (skills/experience) will be displayed to customers but may not be chosen for this opportunity. Actual Years Experience Years Experience Needed Required/ Preferred Skills/Experience 8 Required Implement ETL/ELT workflows for both structured and unstructured data 8 Required Collaborate with cross-functional teams including data scientists, analysts, and stakeholders 8 Required Design and maintain data models, schemas, and database structures to support analytical and operational use cases 8 Required Implement data validation and quality checks to ensure accuracy and consistency 8 Required Contribute to data governance initiatives, including metadata management, data lineage, and data cataloging 8 Required Implement data security measures, including encryption, access controls, and auditing; ensure compliance with regulations and best practices 8 Required Working in agile, multicultural environments 4 Required Automate deployments using CI/CD tools 4 Required Evaluate and implement appropriate data storage solutions, including data lakes (Azure Data Lake Storage) and data warehouses 4 Required Proficiency in Python and R programming languages 4 Required Strong SQL querying and data manipulation skills 4 Required Experience with Azure cloud platform 4 Required Experience with DevOps, CI/CD pipelines, and version control systems 4 Required Strong troubleshooting and debugging capabilities 3 Required Design and develop scalable data pipelines using Apache Spark on Databricks 3 Required Optimize Spark jobs for performance and cost efficiency 3 Required Integrate Databricks solutions with cloud services (Azure Data Factory) 3 Required Ensure data quality, governance, and security using Unity Catalog or Delta Lake 3 Required Deep understanding of Apache Spark architecture, RDDs, DataFrames, and Spark SQL 3 Required Hands-on experience with Databricks notebooks, clusters, jobs, and Delta Lake 2 Required Build AI accelerators and reusable components to boost development teams productivity. 2 Required Implement Gen AI / LLM application and utilities (Agentic AI, Harness / Prompt engineering, RAG) 2 Required Implement AI application governance, observability and evaluation framework 2 Required Design & Implement vector stores for knowledge bases and agent memory 2 Required Integrate AI applications / utils / tools with enterprise traceability and SIEM tools 2 Required Knowledge of ML libraries (MLflow, Scikit-learn, TensorFlow) 1 Preferred Databricks Certified Associate Developer for Apache Spark 1 Preferred Azure Data Engineer Associate DIA SOFTWARE SOLUTIONS LLC. SANGEETHA| DIA SOFTWARE SOLUTIONS LLC. Direct: One Two Six Eight Six Zero Two Four Nine DIA SOFTWARE SOLUTIONS is an Affirmative Action/Equal Opportunity Employer that supports workplace diversity. All employment decisions are made without regard to race, color, religion, sex, national origin, age, disability, veteran status, marital or family status, sexual orientation, gender identity, or genetic information. All Dia soft staff must be able to demonstrate the legal right to work in the United States. DIA SOFTWARE SOLUTIONS is an E-Verify employer