← Back to history

Pipeline run

2ec7ccc9-2b00-4120-a2dd-aefabc025093

Pipeline LLM cost (USD)
API 1: $0.0084 API 2: $0.0001 API 3: $0.0000 Total: $0.0085

Client output enrichment

v2 Skill cluster · Nature of work · AI index · Tech stack maturity · Evidence · KRA description
role baseline loaded sources · ai_index: jd · nature_of_work: jd · tech_stack_maturity: jd
Nature of work · Data platform optimization and governance
Migrate Redshift/RDS workloads to Databricks Lakehouse by converting SQL to PySpark, building and optimizing Delta Lake pipelines, and designing ACID-compliant models with metadata, schema evolution, and lineage. Partner on target architecture, CI/CD, and governance.
""Manage metadata, schema evolution and lineage for migrated datasets""
Tech stack maturity
Modern Cloud Native
The combination of Databricks, Delta Lake, lakehouse architecture, Apache Spark, and CI/CD is characteristic of a modern cloud-native data engineering stack.
AI index (0 = no AI use, 5 = totally AI-dependent · v2.1)
0.50 / 5
· Title match
Has AI skill
· AI skill (primary)
AI skill (secondary)
· On AI team
· Builds AI products
vocab breakdown (legacy)
Assistants (×1):
Frameworks (×2):
Models / concepts (×3): AI, Machine Learning
Evidence — skills matched in JD (14)
Databricks Apache Spark AWS Redshift Amazon RDS Lakehouse ETL SQL PySpark Delta Lake ACID CI/CD Unity Catalog Machine Learning AI
Skill cluster (5 dimension groups, role-scoped)
AI Governance and Model Security
Machine Learning
Data Lineage and Metadata
Unity Catalog
ETL and ELT Tooling
Apache Spark
Programming Languages for Data Work
SQL
Cross-cutting / unaligned
Databricks AWS Redshift Amazon RDS Lakehouse ETL PySpark Delta Lake ACID CI/CD AI
Show KRA description ↓
1. 8+ years with strong Databricks, Spark and Data Migration experience 2. Should have experience of end-to-end migration from AWS Redshift or other RDS to Databricks Lakehouse Platform. Need to have knowledge in Cloud Data Engineering, ETL Modernization, Spark Optimization and SQL-to-PySpark migration. a) Migration Strategy and Execution: analyze the existing DWH/DB architecture, Design & Implement scalable data pipelines in Databricks, SQL Script conversion, etc. b) Development & Optimization: Build Delta Lake tables and Implement ACID-complaint data models, Manage metadata, schema evolution and lineage for migrated datasets c) Collaboration & Governance: Work with Data Architect to define Target data models and migration approaches, Collaborate with DevOps teams to implement CI/CD pipelines, Data Governance teams on Security, lineage and cataloging; Mentor junior members in the team d) Nice to have experience with Unity Catalog for governance and access control; Exposure to machine learning/AI pipelines, etc.

Signals

Skill data-engineer
0.18
Alias data-engineer
1.00
KRA data-engineer
0.58

Post-classification

Centroidupdated · n=163
Alias collision log
New-role queue
New skills captured5
New KRA captured

Captured for admin review

AWS Redshift primary Data Engineer pending
Amazon RDS primary Data Engineer pending
ETL primary Data Engineer pending
PySpark primary Data Engineer pending
ACID primary Data Engineer pending
Status: completed Created: 2026-05-27T14:22:39.141020Z Updated: 2026-05-27T14:24:03.161030Z API 3 duration: 26327 ms
Flow Current 3-step pipeline

1 POST /skills/extract-from-jd

2 POST /skills/extract-details

3 POST /skills/final-role-output

Role Chosen role & resolution

Data Engineer

domain · Data Engineering & Analytics CASE DOMAIN

slug: data-engineer · id: 2 · source: db

Domain=Data Engineering & Analytics; The JD centers on Databricks/Spark-based data migration, pipeline development, ETL modernization, optimization, and cloud data engineering, which best matches Data Engineer.

Matched skills

DatabricksSparkData MigrationAWS RedshiftRDSDatabricks Lakehouse PlatformCloud Data EngineeringETL ModernizationSpark OptimizationSQL-to-PySpark migrationDelta LakeACID-complaint data modelsmetadataschema evolutionUnity Catalog

Matched dimensions

Data migration and modernizationCloud data engineeringDatabricks lakehouse engineeringETL modernization and SQL conversionData pipeline design and implementationSpark performance optimizationData governance and access controlCollaboration with architects and DevOps

Matched KRAs

experience of end-to-end migration from AWS Redshiftanalyze the existing DWH/DB architectureDesign & Implement scalable data pipelines in DatabricksSQL Script conversionBuild Delta Lake tablesManage metadata, schema evolution and lineageCollaborate with DevOps teams to implement CI/CD pipelinesWork with Data Governance teams on Security

Resolution: in_db — role exists in library; skill↔dim and role↔dim links saved when applicable.

0
New skills
0
Skill↔dim saved
0
Role↔dim saved
3
Skipped

Job description

Role : Databricks Senior Developer
Required Technical Skill Set : Databricks, Spark and Data Migration experience
Experience : 4 to 10 years
Location : Kolkata
Job Description : 
Desired Competencies (Technical/Behavioral Competency)
Must-Have:-
1. 8+ years with strong Databricks, Spark and Data Migration experience
2. Should have experience of end-to-end migration from AWS Redshift or other RDS to Databricks Lakehouse Platform. Need to have knowledge in Cloud Data Engineering, ETL Modernization, Spark Optimization and SQL-to-PySpark migration.
3. Key Responsibilities:-
a) Migration Strategy and Execution: analyze the existing DWH/DB architecture, Design & Implement scalable data pipelines in Databricks, SQL Script conversion, etc.
b) Development & Optimization: Build Delta Lake tables and Implement ACID-complaint data models, Manage metadata, schema evolution and lineage for migrated datasets
c) Collaboration & Governance: Work with Data Architect to define Target data models and migration approaches, Collaborate with DevOps teams to implement CI/CD pipelines, Data Governance teams on Security, lineage and cataloging; Mentor junior members in the team
d) Nice to have experience with Unity Catalog for governance and access control; Exposure to machine learning/AI pipelines, etc.

Skills from this JD

Each row merges API 1 extraction, API 2 library match / v3 orchestration (dimensions + locked dims), and API 3 persistence tags.

Databricks Primary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Databricks id=1202 · databricks

Aliases — catalog

  • Databricks (CANONICAL)

Context tags (catalog)

Apache Spark Databricks Runtime Delta Lake MLflow SQL Analytics Spark cloud integration collaborative workspace data engineering data lakes data pipelines data visualization job scheduling machine learning notebooks real-time analytics

Stored enrichment (catalog DB)

Category
Platform
Sub-category
Data Analytics Platform
Vendor
Databricks, Inc.
License
other_open
Year introduced
2013
Confidence
0.97
Version strategy
NOT_APPLICABLE

Maturity reasoning: Databricks appears frequently in data engineering and analytics job postings, especially alongside Spark, Delta Lake, and lakehouse stacks; strong vendor adoption and broad enterprise usage signal mainstream demand.

Skill profile (library / DB)

Skill nature
PLATFORM
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
9
Sub-category id
911
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • React Frontend Development Catalog dimension db id 96

    Library dimension (catalog)

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
React Frontend Development
d_init_01
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Apache Spark Primary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Apache Spark id=1350 · apache-spark

Aliases — catalog

  • Apache Spark (CANONICAL)
  • apache spark 3 (VERSION)
  • spark (VERSION)
  • spark 3 (VERSION)
  • spark 3.x (VERSION)
  • spark3 (VERSION)

Context tags (catalog)

Apache Kafka Cluster Manager DAGScheduler Data Lake DataFrame ETL Hadoop MLlib Machine Learning PySpark RDD Scala Spark SQL Spark Streaming SparkSession

Stored enrichment (catalog DB)

Category
Framework
Sub-category
Distributed Data Processing Framework
Vendor
Apache Software Foundation
License
apache_2
Year introduced
2010
Confidence
0.94
Version strategy
SEPARATE_ENTITY
Version tag
3.x

Maturity reasoning: Apache Spark appears in many data engineering JDs and remains a standard for distributed ETL/ELT; its GitHub and vendor ecosystem activity stay strong, with Databricks and cloud platforms still promoting it.

Skill profile (library / DB)

Skill nature
FRAMEWORK
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
5
Sub-category id
1021
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • ETL and ELT Tooling Catalog dimension db id 24

    Library dimension (catalog)

    Roles linked in library: Data Engineer

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
ETL and ELT Tooling
etl-and-elt-tooling
Existing dimension (library) · Role↔dimension saved
AWS Redshift Primary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Amazon Redshift id=107 · amazon-redshift

Aliases — catalog

  • Amazon Redshift (CANONICAL) primary

Context tags (catalog)

AWS Glue Amazon S3 BI COPY command ELT ETL JDBC ODBC RA3 SQL Spectrum analytics data warehouse distribution key sort key

Stored enrichment (catalog DB)

Category
Service
Sub-category
Data Warehouse Service
Vendor
Amazon Web Services
License
proprietary
Year introduced
2012
Confidence
0.97
Version strategy
NOT_APPLICABLE

Maturity reasoning: Commonly listed in data/analytics job descriptions and widely used as AWS’s managed warehouse; strong vendor adoption and steady JD volume signal broad market demand.

Skill profile (library / DB)

Skill nature
CLOUD_SERVICE
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
11
Sub-category id
118
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • Cloud Data Warehouses Catalog dimension db id 22

    Library dimension (catalog)

    Roles linked in library: Data Engineer

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
Cloud Data Warehouses
cloud-data-warehouses
Skipped — no persistable v3 meta for new skill
skill_not_in_db_v3_proposed
Amazon RDS Primary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: AWS RDS id=3632 · aws-rds

Aliases — catalog

  • AWS RDS (CANONICAL) primary

Context tags (catalog)

Aurora IAM roles MySQL PostgreSQL backup and restore cloud formation database migration encryption monitoring multi-AZ parameter groups performance tuning read replicas scalability security groups

Stored enrichment (catalog DB)

Category
Service
Sub-category
Managed Relational Database Service
Vendor
Amazon Web Services
License
proprietary
Year introduced
2009
Confidence
0.98
Version strategy
NOT_APPLICABLE

Maturity reasoning: AWS RDS is broadly listed in cloud/backend job descriptions and is a standard managed database service in AWS stacks; it remains actively offered by AWS with no sunset signal.

Skill profile (library / DB)

Skill nature
CLOUD_SERVICE
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
11
Sub-category id
2845
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • Cloud Platforms & Hosting Providers Catalog dimension db id 278

    Library dimension (catalog)

    Roles linked in library: .NET Backend Developer, Kotlin Backend Developer, Scala Backend Developer, Web Developer

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
Cloud Platforms & Hosting Providers
cloud-platforms-hosting-providers
Skipped — no persistable v3 meta for new skill
skill_not_in_db_v3_proposed
Lakehouse Primary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Lakehouse id=1359 · lakehouse

Aliases — catalog

  • Lakehouse (CANONICAL)

Context tags (catalog)

Apache Spark Delta Lake ETL SQL analytics cloud storage data governance data integration data lake data modeling data pipeline data warehouse metadata management real-time processing streaming analytics

Stored enrichment (catalog DB)

Category
Architecture
Sub-category
Data Platform Architecture
Confidence
0.90
Version strategy
NOT_APPLICABLE

Maturity reasoning: Lakehouse is increasingly listed in data-platform JDs and vendor docs (Databricks, Snowflake, Microsoft Fabric), but it is not yet as universal as core warehouse or lake skills.

Skill profile (library / DB)

Skill nature
PATTERN
Volatility
EMERGING
Typical lifespan
EVERGREEN
Category id
1
Sub-category id
1026
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • React Frontend Development Catalog dimension db id 96

    Library dimension (catalog)

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
React Frontend Development
d_init_01
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
ETL Primary New / orchestrated API 3: new canonical path (new) New / unmatched skill (orchestrated in API 2)

Skill enrichment (orchestrator / LLM)

No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).

Derived legacy fields
Category
Data Engineering Tools
Sub-category
general
Skill nature
PRACTICE
Volatility
MEDIUM
Typical lifespan
MULTI_YEAR
Version strategy
UNVERSIONED
SQL Primary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: SQL id=101 · sql

Aliases — catalog

  • SQL (CANONICAL) primary

Context tags (catalog)

ACID CTE DDL DML ETL JOIN MySQL NoSQL OLAP ORM PostgreSQL SQL injection SQLite T-SQL data modeling data warehousing database normalization execution plan indexing joins normalization query optimization stored procedures subquery transaction isolation transaction management window functions

Stored enrichment (catalog DB)

Category
Language
Sub-category
Query Language
Vendor
ANSI
License
unknown
Year introduced
1974
Confidence
0.99
Version strategy
NOT_APPLICABLE

Maturity reasoning: SQL appears in a large share of data, backend, and analytics job descriptions and remains the default query language for PostgreSQL, MySQL, and cloud warehouses like Snowflake/BigQuery.

Skill profile (library / DB)

Skill nature
LANGUAGE
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
6
Sub-category id
97
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • Pega Programming Languages & DSLs Catalog dimension db id 267

    Library dimension (catalog)

    Roles linked in library: Pega Developer

  • Programming Languages for Data Work Catalog dimension db id 21

    Library dimension (catalog)

    Roles linked in library: Data Engineer

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
Pega Programming Languages & DSLs
pega-programming-languages-dsls
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Programming Languages for Data Work
programming-languages-for-data-work
Existing dimension (library) · Role↔dimension saved
PySpark Primary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Apache Spark id=1350 · apache-spark

Aliases — catalog

  • Apache Spark (CANONICAL)
  • apache spark 3 (VERSION)
  • spark (VERSION)
  • spark 3 (VERSION)
  • spark 3.x (VERSION)
  • spark3 (VERSION)

Context tags (catalog)

Apache Kafka Cluster Manager DAGScheduler Data Lake DataFrame ETL Hadoop MLlib Machine Learning PySpark RDD Scala Spark SQL Spark Streaming SparkSession

Stored enrichment (catalog DB)

Category
Framework
Sub-category
Distributed Data Processing Framework
Vendor
Apache Software Foundation
License
apache_2
Year introduced
2010
Confidence
0.94
Version strategy
SEPARATE_ENTITY
Version tag
3.x

Maturity reasoning: Apache Spark appears in many data engineering JDs and remains a standard for distributed ETL/ELT; its GitHub and vendor ecosystem activity stay strong, with Databricks and cloud platforms still promoting it.

Skill profile (library / DB)

Skill nature
FRAMEWORK
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
5
Sub-category id
1021
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • ETL and ELT Tooling Catalog dimension db id 24

    Library dimension (catalog)

    Roles linked in library: Data Engineer

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
ETL and ELT Tooling
etl-and-elt-tooling
Skipped — no persistable v3 meta for new skill
skill_not_in_db_v3_proposed
Delta Lake Primary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Delta Lake id=237 · delta-lake

Aliases — catalog

  • Delta Lake (CANONICAL) primary

Context tags (catalog)

ACID transactions Apache Spark CDC Databricks Delta Engine ETL Lakehouse architecture MERGE INTO OPTIMIZE Parquet Unity Catalog Z-Order batch processing cloud storage data governance data lake data lakehouse data pipeline data reliability partition pruning schema enforcement schema evolution streaming data streaming ingestion time travel

Stored enrichment (catalog DB)

Category
Tool
Sub-category
Table Format Tool
Vendor
Databricks
License
apache_2
Year introduced
2017
Confidence
0.72
Version strategy
NOT_APPLICABLE

Maturity reasoning: Delta Lake appears frequently in data engineering JDs and cloud vendor docs, especially alongside Databricks/Spark for lakehouse stacks; it’s a common hiring-pipeline skill rather than a niche tool.

Skill profile (library / DB)

Skill nature
TOOL
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
13
Sub-category id
1170
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • Model and Data Versioning Catalog dimension db id 48

    Library dimension (catalog)

    Roles linked in library: ML Engineer, MLOps Engineer

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
Model and Data Versioning
model-and-data-versioning
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
ACID Primary New / orchestrated API 3: new canonical path (new) New / unmatched skill (orchestrated in API 2)

Skill enrichment (orchestrator / LLM)

No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).

Derived legacy fields
Category
Concepts
Sub-category
general
Skill nature
CONCEPT
Volatility
STABLE
Typical lifespan
EVERGREEN
Version strategy
UNVERSIONED
CI/CD Primary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: CI/CD id=1190 · ci-cd

Aliases — catalog

  • CI/CD (CANONICAL)

Context tags (catalog)

Ansible CircleCI Docker GitLab CI Jenkins Kubernetes Terraform Travis CI automated testing build automation continuous deployment continuous integration deployment pipelines monitoring version control

Stored enrichment (catalog DB)

Category
Methodology
Sub-category
Ci Cd Process
Confidence
0.93
Version strategy
NOT_APPLICABLE

Maturity reasoning: CI/CD appears in a large share of software engineering JDs and is a standard requirement across DevOps, platform, and backend roles; major vendors like GitHub, GitLab, and AWS all center product roadmaps on CI/CD pipelines.

Skill profile (library / DB)

Skill nature
METHODOLOGY
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
8
Sub-category id
900
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • CI/CD Pipeline Platforms Catalog dimension db id 150

    Library dimension (catalog)

    Roles linked in library: DevOps Engineer

  • CI/CD for Machine Learning Catalog dimension db id 56

    Library dimension (catalog)

    Roles linked in library: ML Engineer

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
CI/CD Pipeline Platforms
ci-cd-pipeline-platforms
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
CI/CD for Machine Learning
ci-cd-for-machine-learning
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Unity Catalog Secondary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Unity Catalog id=1203 · unity-catalog

Aliases — catalog

  • Unity Catalog (CANONICAL)

Context tags (catalog)

access control audit logs collaboration compliance data catalog data classification data discovery data governance data lineage data privacy data quality data sharing metadata management role-based access schema enforcement

Stored enrichment (catalog DB)

Category
Platform
Sub-category
Data Governance Platform
Vendor
Databricks
License
unknown
Year introduced
2021
Confidence
0.90
Version strategy
NOT_APPLICABLE

Maturity reasoning: Appears increasingly in Databricks and data-governance job postings, but JD volume is still far below core platforms like AWS or PostgreSQL; adoption is growing with Unity Catalog positioned as Databricks’ unified governance layer.

Skill profile (library / DB)

Skill nature
PLATFORM
Volatility
EMERGING
Typical lifespan
EVERGREEN
Category id
9
Sub-category id
250
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • Data Lineage and Metadata Catalog dimension db id 28

    Library dimension (catalog)

    Roles linked in library: Data Engineer

  • React Frontend Development Catalog dimension db id 96

    Library dimension (catalog)

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
Data Lineage and Metadata
data-lineage-and-metadata
Existing dimension (library) · Role↔dimension saved
React Frontend Development
d_init_01
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Machine Learning Secondary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Machine Learning id=1356 · machine-learning

Aliases — catalog

  • Machine Learning (CANONICAL)

Context tags (catalog)

Keras PyTorch TensorFlow cross-validation data preprocessing ensemble methods feature engineering hyperparameter tuning model evaluation natural language processing neural networks reinforcement learning scikit-learn supervised learning unsupervised learning

Stored enrichment (catalog DB)

Category
Concept
Sub-category
Machine Learning
Confidence
0.98
Version strategy
NOT_APPLICABLE

Maturity reasoning: Machine Learning appears in large volumes of job descriptions across data, product, and platform roles, and major cloud vendors (AWS, Google Cloud, Azure) offer dedicated ML services and certifications, indicating broad adoption.

Skill profile (library / DB)

Skill nature
CONCEPT
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
2
Sub-category id
1024
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • AI Governance and Model Security Catalog dimension db id 50

    Library dimension (catalog)

    Roles linked in library: AI Engineer, ML Engineer, MLOps Engineer

  • React Frontend Development Catalog dimension db id 96

    Library dimension (catalog)

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
AI Governance and Model Security
ai-governance-and-model-security
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
React Frontend Development
d_init_01
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
AI Secondary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: AI id=1347 · ai

Aliases — catalog

  • AI (CANONICAL)

Context tags (catalog)

AI ethics PyTorch TensorFlow algorithm optimization computer vision data preprocessing deep learning feature engineering machine learning model training natural language processing neural networks predictive analytics reinforcement learning supervised learning

Stored enrichment (catalog DB)

Category
Concept
Sub-category
Artificial Intelligence
Confidence
0.98
Version strategy
NOT_APPLICABLE

Maturity reasoning: AI appears in a large and growing share of job descriptions across software, data, and product roles; major vendors like Microsoft, Google, and AWS have broad AI offerings and hiring demand reflects mainstream adoption.

Skill profile (library / DB)

Skill nature
CONCEPT
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
2
Sub-category id
1020
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • React Frontend Development Catalog dimension db id 96

    Library dimension (catalog)

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
React Frontend Development
d_init_01
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)

All API 3 persistence rows

Same grid as the skill-extractor “Persistence items” table: one row per (skill × dimension) work item.

Skill Tag Dimension Skill↔dim Role↔dim Outcome Notes
Databricks in_db
React Frontend Development
d_init_01
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Apache Spark in_db
ETL and ELT Tooling
etl-and-elt-tooling
Existing dimension (library) · Role↔dimension saved
AWS Redshift new
Cloud Data Warehouses
cloud-data-warehouses
Skipped — no persistable v3 meta for new skill skill_not_in_db_v3_proposed
Amazon RDS new
Cloud Platforms & Hosting Providers
cloud-platforms-hosting-providers
Skipped — no persistable v3 meta for new skill skill_not_in_db_v3_proposed
Lakehouse in_db
React Frontend Development
d_init_01
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
SQL in_db
Pega Programming Languages & DSLs
pega-programming-languages-dsls
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
SQL in_db
Programming Languages for Data Work
programming-languages-for-data-work
Existing dimension (library) · Role↔dimension saved
PySpark new
ETL and ELT Tooling
etl-and-elt-tooling
Skipped — no persistable v3 meta for new skill skill_not_in_db_v3_proposed
Delta Lake in_db
Model and Data Versioning
model-and-data-versioning
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
CI/CD in_db
CI/CD Pipeline Platforms
ci-cd-pipeline-platforms
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
CI/CD in_db
CI/CD for Machine Learning
ci-cd-for-machine-learning
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Unity Catalog in_db
Data Lineage and Metadata
data-lineage-and-metadata
Existing dimension (library) · Role↔dimension saved
Unity Catalog in_db
React Frontend Development
d_init_01
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Machine Learning in_db
AI Governance and Model Security
ai-governance-and-model-security
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Machine Learning in_db
React Frontend Development
d_init_01
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
AI in_db
React Frontend Development
d_init_01
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)

Library artifacts (this run)

Kind Detail DB id
canonical_skill_proposed ETL | type=Data Engineering Tools subtype=general nature=PRACTICE lifespan=MULTI_YEAR
canonical_skill_proposed ACID | type=Concepts subtype=general nature=CONCEPT lifespan=EVERGREEN
dimension_skill_link_proposed AWS Redshift ↔ Cloud Data Warehouses
role_dimension_link_proposed Data Engineer ↔ Cloud Data Warehouses
dimension_skill_link_proposed Amazon RDS ↔ Cloud Platforms & Hosting Providers
dimension_skill_link_proposed PySpark ↔ ETL and ELT Tooling
role_dimension_link_proposed Data Engineer ↔ ETL and ELT Tooling
nano JD Parser — gpt-4.1-nano click to toggle
RoleDatabricks Senior Developer
Experience4 to 10 years
DomainIT Services & Consulting
Location Kolkata, India
JD type pass
Show raw JSON
{
  "JD_type": "pass",
  "about_company": null,
  "certifications": [],
  "company_name": null,
  "ctc": null,
  "domain": {
    "primary": {
      "aliases": [],
      "domain": "IT Services \u0026 Consulting"
    },
    "secondary": null
  },
  "education": [],
  "experience": {
    "max": 10,
    "min": 4,
    "raw": "4 to 10 years"
  },
  "job_locations": [
    {
      "aliases": [
        "Calcutta"
      ],
      "city": "Kolkata",
      "country": "India",
      "state": null,
      "work_mode": null
    }
  ],
  "role": "Databricks Senior Developer",
  "role_aliases": [
    "Senior Databricks Developer",
    "Databricks Developer",
    "Data Engineer"
  ],
  "role_archetype": "Data",
  "roles_and_responsibilities": [
    {
      "bullet_count": 2,
      "heading": "Must-Have",
      "heading_was_present": true,
      "source_marker": {
        "first_5_words": "1. 8+ years with strong",
        "last_5_words": "SQL-to-PySpark migration."
      },
      "text": "1. 8+ years with strong Databricks, Spark and Data Migration experience\n2. Should have experience of end-to-end migration from AWS Redshift or other RDS to Databricks Lakehouse Platform. Need to have knowledge in Cloud Data Engineering, ETL Modernization, Spark Optimization and SQL-to-PySpark migration.",
      "word_count": 47
    },
    {
      "bullet_count": 4,
      "heading": "Key Responsibilities",
      "heading_was_present": true,
      "source_marker": {
        "first_5_words": "a) Migration Strategy and Execution:",
        "last_5_words": "machine learning/AI pipelines, etc."
      },
      "text": "a) Migration Strategy and Execution: analyze the existing DWH/DB architecture, Design \u0026 Implement scalable data pipelines in Databricks, SQL Script conversion, etc.\nb) Development \u0026 Optimization: Build Delta Lake tables and Implement ACID-complaint data models, Manage metadata, schema evolution and lineage for migrated datasets\nc) Collaboration \u0026 Governance: Work with Data Architect to define Target data models and migration approaches, Collaborate with DevOps teams to implement CI/CD pipelines, Data Governance teams on Security, lineage and cataloging; Mentor junior members in the team\nd) Nice to have experience with Unity Catalog for governance and access control; Exposure to machine learning/AI pipelines, etc.",
      "word_count": 104
    }
  ],
  "urls": []
}
API 1 — extract-from-jd click to toggle
{
  "final_skills": [
    {
      "is_primary": true,
      "skill_name": "Databricks"
    },
    {
      "is_primary": true,
      "skill_name": "Apache Spark"
    },
    {
      "is_primary": true,
      "skill_name": "AWS Redshift"
    },
    {
      "is_primary": true,
      "skill_name": "Amazon RDS"
    },
    {
      "is_primary": true,
      "skill_name": "Lakehouse"
    },
    {
      "is_primary": true,
      "skill_name": "ETL"
    },
    {
      "is_primary": true,
      "skill_name": "SQL"
    },
    {
      "is_primary": true,
      "skill_name": "PySpark"
    },
    {
      "is_primary": true,
      "skill_name": "Delta Lake"
    },
    {
      "is_primary": true,
      "skill_name": "ACID"
    },
    {
      "is_primary": true,
      "skill_name": "CI/CD"
    },
    {
      "is_primary": false,
      "skill_name": "Unity Catalog"
    },
    {
      "is_primary": false,
      "skill_name": "Machine Learning"
    },
    {
      "is_primary": false,
      "skill_name": "AI"
    }
  ],
  "jd_role": {
    "display_name": "Databricks Senior Developer",
    "rationale": null,
    "role_aliases": [
      "Senior Databricks Developer",
      "Databricks Developer",
      "Data Engineer"
    ],
    "role_archetype": "Data",
    "slug": ""
  },
  "nano_parsed": {
    "JD_type": "pass",
    "about_company": null,
    "certifications": [],
    "company_name": null,
    "ctc": null,
    "domain": {
      "primary": {
        "aliases": [],
        "domain": "IT Services \u0026 Consulting"
      },
      "secondary": null
    },
    "education": [],
    "experience": {
      "max": 10,
      "min": 4,
      "raw": "4 to 10 years"
    },
    "job_locations": [
      {
        "aliases": [
          "Calcutta"
        ],
        "city": "Kolkata",
        "country": "India",
        "state": null,
        "work_mode": null
      }
    ],
    "role": "Databricks Senior Developer",
    "role_aliases": [
      "Senior Databricks Developer",
      "Databricks Developer",
      "Data Engineer"
    ],
    "role_archetype": "Data",
    "roles_and_responsibilities": [
      {
        "bullet_count": 2,
        "heading": "Must-Have",
        "heading_was_present": true,
        "source_marker": {
          "first_5_words": "1. 8+ years with strong",
          "last_5_words": "SQL-to-PySpark migration."
        },
        "text": "1. 8+ years with strong Databricks, Spark and Data Migration experience\n2. Should have experience of end-to-end migration from AWS Redshift or other RDS to Databricks Lakehouse Platform. Need to have knowledge in Cloud Data Engineering, ETL Modernization, Spark Optimization and SQL-to-PySpark migration.",
        "word_count": 47
      },
      {
        "bullet_count": 4,
        "heading": "Key Responsibilities",
        "heading_was_present": true,
        "source_marker": {
          "first_5_words": "a) Migration Strategy and Execution:",
          "last_5_words": "machine learning/AI pipelines, etc."
        },
        "text": "a) Migration Strategy and Execution: analyze the existing DWH/DB architecture, Design \u0026 Implement scalable data pipelines in Databricks, SQL Script conversion, etc.\nb) Development \u0026 Optimization: Build Delta Lake tables and Implement ACID-complaint data models, Manage metadata, schema evolution and lineage for migrated datasets\nc) Collaboration \u0026 Governance: Work with Data Architect to define Target data models and migration approaches, Collaborate with DevOps teams to implement CI/CD pipelines, Data Governance teams on Security, lineage and cataloging; Mentor junior members in the team\nd) Nice to have experience with Unity Catalog for governance and access control; Exposure to machine learning/AI pipelines, etc.",
        "word_count": 104
      }
    ],
    "urls": []
  },
  "rejected": false,
  "rejection_reason": null,
  "run_id": "2ec7ccc9-2b00-4120-a2dd-aefabc025093",
  "stage3_signals": {
    "alias_found": true,
    "alias_match_roles": [
      {
        "display_name": "Data Engineer",
        "kra_matches": null,
        "matched_count": null,
        "matched_skills": null,
        "role_id": 2,
        "score": 1.0,
        "slug": "data-engineer",
        "total_count": null
      },
      {
        "display_name": "Data Warehouse Engineer",
        "kra_matches": null,
        "matched_count": null,
        "matched_skills": null,
        "role_id": 144,
        "score": 1.0,
        "slug": "data-warehouse-engineer",
        "total_count": null
      }
    ],
    "kra_match_roles": [
      {
        "display_name": "Data Engineer",
        "kra_matches": [
          {
            "kra_text": "Works with data analysts, data scientists, and business stakeholders to define data models, ingestion schedules, and data delivery requirements.",
            "sentence": "c) Collaboration \u0026 Governance: Work with Data Architect to define Target data models and migration approaches, Collaborate with DevOps teams to implement CI/CD pipelines, Data Governance teams on Security, lineage and cataloging;",
            "similarity": 0.616
          },
          {
            "kra_text": "Optimizes pipeline throughput, partitioning strategies, and query performance across cloud data warehouses like Snowflake, BigQuery, or Redshift.",
            "sentence": "a) Migration Strategy and Execution: analyze the existing DWH/DB architecture, Design \u0026 Implement scalable data pipelines in Databricks, SQL Script conversion, etc. b) Development \u0026 Optimization: Build Delta Lake tables and Implement ACID-complaint data models, Manage metadata, schema evolution and lineage for migrated datasets",
            "similarity": 0.5647
          },
          {
            "kra_text": "Develops batch and real-time streaming data pipelines using Apache Spark, Apache Kafka, Apache Flink, or Airflow for data movement and processing at scale.",
            "sentence": "Need to have knowledge in Cloud Data Engineering, ETL Modernization, Spark Optimization and SQL-to-PySpark migration.",
            "similarity": 0.5579
          }
        ],
        "matched_count": null,
        "matched_skills": null,
        "role_id": 2,
        "score": 0.5795,
        "slug": "data-engineer",
        "total_count": null
      },
      {
        "display_name": "Cloud Architect",
        "kra_matches": [
          {
            "kra_text": "Conducts architecture reviews, approves technical design documents, and guides engineering teams through cloud migration and modernization projects.",
            "sentence": "c) Collaboration \u0026 Governance: Work with Data Architect to define Target data models and migration approaches, Collaborate with DevOps teams to implement CI/CD pipelines, Data Governance teams on Security, lineage and cataloging;",
            "similarity": 0.5154
          },
          {
            "kra_text": "Defines cloud adoption roadmaps, lift-and-shift vs. refactor migration strategies, and landing zone architectures for workloads moving to AWS, Azure, or GCP.",
            "sentence": "a) Migration Strategy and Execution: analyze the existing DWH/DB architecture, Design \u0026 Implement scalable data pipelines in Databricks, SQL Script conversion, etc. b) Development \u0026 Optimization: Build Delta Lake tables and Implement ACID-complaint data models, Manage metadata, schema evolution and lineage for migrated datasets",
            "similarity": 0.5083
          },
          {
            "kra_text": "Defines cloud adoption roadmaps, lift-and-shift vs. refactor migration strategies, and landing zone architectures for workloads moving to AWS, Azure, or GCP.",
            "sentence": "Need to have knowledge in Cloud Data Engineering, ETL Modernization, Spark Optimization and SQL-to-PySpark migration.",
            "similarity": 0.4411
          }
        ],
        "matched_count": null,
        "matched_skills": null,
        "role_id": 9,
        "score": 0.4883,
        "slug": "cloud-architect",
        "total_count": null
      },
      {
        "display_name": "Fullstack Developer",
        "kra_matches": [
          {
            "kra_text": "Designs and queries relational databases like PostgreSQL and document stores like MongoDB, writing migrations, indexes, and optimized queries.",
            "sentence": "a) Migration Strategy and Execution: analyze the existing DWH/DB architecture, Design \u0026 Implement scalable data pipelines in Databricks, SQL Script conversion, etc. b) Development \u0026 Optimization: Build Delta Lake tables and Implement ACID-complaint data models, Manage metadata, schema evolution and lineage for migrated datasets",
            "similarity": 0.4943
          },
          {
            "kra_text": "Designs and queries relational databases like PostgreSQL and document stores like MongoDB, writing migrations, indexes, and optimized queries.",
            "sentence": "8+ years with strong Databricks, Spark and Data Migration experience",
            "similarity": 0.4122
          },
          {
            "kra_text": "Designs and queries relational databases like PostgreSQL and document stores like MongoDB, writing migrations, indexes, and optimized queries.",
            "sentence": "Need to have knowledge in Cloud Data Engineering, ETL Modernization, Spark Optimization and SQL-to-PySpark migration.",
            "similarity": 0.4116
          }
        ],
        "matched_count": null,
        "matched_skills": null,
        "role_id": 15,
        "score": 0.4394,
        "slug": "full-stack-engineer",
        "total_count": null
      },
      {
        "display_name": "Svelte Frontend Developer",
        "kra_matches": [
          {
            "kra_text": "backend data integration",
            "sentence": "a) Migration Strategy and Execution: analyze the existing DWH/DB architecture, Design \u0026 Implement scalable data pipelines in Databricks, SQL Script conversion, etc. b) Development \u0026 Optimization: Build Delta Lake tables and Implement ACID-complaint data models, Manage metadata, schema evolution and lineage for migrated datasets",
            "similarity": 0.4761
          },
          {
            "kra_text": "backend data integration",
            "sentence": "c) Collaboration \u0026 Governance: Work with Data Architect to define Target data models and migration approaches, Collaborate with DevOps teams to implement CI/CD pipelines, Data Governance teams on Security, lineage and cataloging;",
            "similarity": 0.4402
          },
          {
            "kra_text": "backend data integration",
            "sentence": "Should have experience of end-to-end migration from AWS Redshift or other RDS to Databricks Lakehouse Platform.",
            "similarity": 0.3938
          }
        ],
        "matched_count": null,
        "matched_skills": null,
        "role_id": 92,
        "score": 0.4367,
        "slug": "svelte-frontend-developer",
        "total_count": null
      },
      {
        "display_name": "Java Backend Developer",
        "kra_matches": [
          {
            "kra_text": "persistence and data modeling",
            "sentence": "c) Collaboration \u0026 Governance: Work with Data Architect to define Target data models and migration approaches, Collaborate with DevOps teams to implement CI/CD pipelines, Data Governance teams on Security, lineage and cataloging;",
            "similarity": 0.4716
          },
          {
            "kra_text": "persistence and data modeling",
            "sentence": "a) Migration Strategy and Execution: analyze the existing DWH/DB architecture, Design \u0026 Implement scalable data pipelines in Databricks, SQL Script conversion, etc. b) Development \u0026 Optimization: Build Delta Lake tables and Implement ACID-complaint data models, Manage metadata, schema evolution and lineage for migrated datasets",
            "similarity": 0.4319
          },
          {
            "kra_text": "persistence and data modeling",
            "sentence": "8+ years with strong Databricks, Spark and Data Migration experience",
            "similarity": 0.402
          }
        ],
        "matched_count": null,
        "matched_skills": null,
        "role_id": 79,
        "score": 0.4352,
        "slug": "java-backend-developer",
        "total_count": null
      }
    ],
    "skill_match_roles": [
      {
        "display_name": "Data Engineer",
        "kra_matches": null,
        "matched_count": 2,
        "matched_skills": [
          "Apache Spark",
          "SQL"
        ],
        "role_id": 2,
        "score": 0.1818,
        "slug": "data-engineer",
        "total_count": 11
      },
      {
        "display_name": "ML Engineer",
        "kra_matches": null,
        "matched_count": 2,
        "matched_skills": [
          "CI/CD",
          "Delta Lake"
        ],
        "role_id": 3,
        "score": 0.1818,
        "slug": "ml-engineer",
        "total_count": 11
      },
      {
        "display_name": "DevOps Engineer",
        "kra_matches": null,
        "matched_count": 1,
        "matched_skills": [
          "CI/CD"
        ],
        "role_id": 10,
        "score": 0.0909,
        "slug": "devops-engineer",
        "total_count": 11
      },
      {
        "display_name": "MLOps Engineer",
        "kra_matches": null,
        "matched_count": 1,
        "matched_skills": [
          "Delta Lake"
        ],
        "role_id": 16,
        "score": 0.0909,
        "slug": "ml-ops-engineer",
        "total_count": 11
      },
      {
        "display_name": "Pega Developer",
        "kra_matches": null,
        "matched_count": 1,
        "matched_skills": [
          "SQL"
        ],
        "role_id": 24,
        "score": 0.0909,
        "slug": "pega-developer",
        "total_count": 11
      }
    ]
  },
  "stage4_decision": {
    "alias_collision_detected": false,
    "case": "DOMAIN",
    "chosen_role": {
      "display_name": "Data Engineer",
      "kra_matches": null,
      "matched_count": null,
      "matched_skills": null,
      "role_id": 2,
      "score": 0.98,
      "slug": "data-engineer",
      "total_count": null
    },
    "confidence": 0.98,
    "is_new_role": false,
    "llm2_fired": false,
    "llm2_reasoning": null,
    "matched_dimensions": [
      "Data migration and modernization",
      "Cloud data engineering",
      "Databricks lakehouse engineering",
      "ETL modernization and SQL conversion",
      "Data pipeline design and implementation",
      "Spark performance optimization",
      "Data governance and access control",
      "Collaboration with architects and DevOps"
    ],
    "matched_kras": [
      "experience of end-to-end migration from AWS Redshift",
      "analyze the existing DWH/DB architecture",
      "Design \u0026 Implement scalable data pipelines in Databricks",
      "SQL Script conversion",
      "Build Delta Lake tables",
      "Manage metadata, schema evolution and lineage",
      "Collaborate with DevOps teams to implement CI/CD pipelines",
      "Work with Data Governance teams on Security"
    ],
    "matched_skills": [
      "Databricks",
      "Spark",
      "Data Migration",
      "AWS Redshift",
      "RDS",
      "Databricks Lakehouse Platform",
      "Cloud Data Engineering",
      "ETL Modernization",
      "Spark Optimization",
      "SQL-to-PySpark migration",
      "Delta Lake",
      "ACID-complaint data models",
      "metadata",
      "schema evolution",
      "Unity Catalog"
    ],
    "new_role_display_name": null,
    "new_role_slug": null,
    "queued": false,
    "reasoning": "Domain=Data Engineering \u0026 Analytics; The JD centers on Databricks/Spark-based data migration, pipeline development, ETL modernization, optimization, and cloud data engineering, which best matches Data Engineer.",
    "sub_role": null
  },
  "stage5_updates": {
    "centroid_n_after": 163,
    "centroid_updated": true,
    "collision_log_id": null,
    "new_kra_attached": null,
    "new_skills_attached": [
      {
        "is_primary": true,
        "queue_id": 8610,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "AWS Redshift",
        "status": "pending"
      },
      {
        "is_primary": true,
        "queue_id": 8611,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "Amazon RDS",
        "status": "pending"
      },
      {
        "is_primary": true,
        "queue_id": 8612,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "ETL",
        "status": "pending"
      },
      {
        "is_primary": true,
        "queue_id": 8613,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "PySpark",
        "status": "pending"
      },
      {
        "is_primary": true,
        "queue_id": 8614,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "ACID",
        "status": "pending"
      }
    ],
    "queue_entry_id": null,
    "v3_pipeline_triggered": false,
    "v3_role_slug": null,
    "v3_run_id": null
  }
}
API 2 — extract-details
{
  "alias_matches": [
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 1838,
      "existing_alias_text": "Databricks",
      "input_term": "Databricks",
      "matched_canonical": {
        "category_id": 9,
        "display_name": "Databricks",
        "id": 1202,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "PLATFORM",
        "slug": "databricks",
        "sub_category_id": 911,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 2004,
      "existing_alias_text": "Apache Spark",
      "input_term": "Apache Spark",
      "matched_canonical": {
        "category_id": 5,
        "display_name": "Apache Spark",
        "id": 1350,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "FRAMEWORK",
        "slug": "apache-spark",
        "sub_category_id": 1021,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "TODO: REMOVE AFTER TESTING \u2014 alias DB write disabled",
      "alias_persisted": false,
      "existing_alias_id": 301,
      "existing_alias_text": "Amazon Redshift",
      "input_term": "AWS Redshift",
      "matched_canonical": {
        "category_id": 11,
        "display_name": "Amazon Redshift",
        "id": 107,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "CLOUD_SERVICE",
        "slug": "amazon-redshift",
        "sub_category_id": 118,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "embedding_alias"
    },
    {
      "alias_persist_skipped_reason": "TODO: REMOVE AFTER TESTING \u2014 alias DB write disabled",
      "alias_persisted": false,
      "existing_alias_id": 5229,
      "existing_alias_text": "AWS RDS",
      "input_term": "Amazon RDS",
      "matched_canonical": {
        "category_id": 11,
        "display_name": "AWS RDS",
        "id": 3632,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "CLOUD_SERVICE",
        "slug": "aws-rds",
        "sub_category_id": 2845,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "embedding_alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 2018,
      "existing_alias_text": "Lakehouse",
      "input_term": "Lakehouse",
      "matched_canonical": {
        "category_id": 1,
        "display_name": "Lakehouse",
        "id": 1359,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "PATTERN",
        "slug": "lakehouse",
        "sub_category_id": 1026,
        "typical_lifespan": "EVERGREEN",
        "volatility": "EMERGING"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 271,
      "existing_alias_text": "SQL",
      "input_term": "SQL",
      "matched_canonical": {
        "category_id": 6,
        "display_name": "SQL",
        "id": 101,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "LANGUAGE",
        "slug": "sql",
        "sub_category_id": 97,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "TODO: REMOVE AFTER TESTING \u2014 alias DB write disabled",
      "alias_persisted": false,
      "existing_alias_id": 2004,
      "existing_alias_text": "Apache Spark",
      "input_term": "PySpark",
      "matched_canonical": {
        "category_id": 5,
        "display_name": "Apache Spark",
        "id": 1350,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "FRAMEWORK",
        "slug": "apache-spark",
        "sub_category_id": 1021,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "embedding_alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 498,
      "existing_alias_text": "Delta Lake",
      "input_term": "Delta Lake",
      "matched_canonical": {
        "category_id": 13,
        "display_name": "Delta Lake",
        "id": 237,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "TOOL",
        "slug": "delta-lake",
        "sub_category_id": 1170,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 1826,
      "existing_alias_text": "CI/CD",
      "input_term": "CI/CD",
      "matched_canonical": {
        "category_id": 8,
        "display_name": "CI/CD",
        "id": 1190,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "METHODOLOGY",
        "slug": "ci-cd",
        "sub_category_id": 900,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 1839,
      "existing_alias_text": "Unity Catalog",
      "input_term": "Unity Catalog",
      "matched_canonical": {
        "category_id": 9,
        "display_name": "Unity Catalog",
        "id": 1203,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "PLATFORM",
        "slug": "unity-catalog",
        "sub_category_id": 250,
        "typical_lifespan": "EVERGREEN",
        "volatility": "EMERGING"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 2015,
      "existing_alias_text": "Machine Learning",
      "input_term": "Machine Learning",
      "matched_canonical": {
        "category_id": 2,
        "display_name": "Machine Learning",
        "id": 1356,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "CONCEPT",
        "slug": "machine-learning",
        "sub_category_id": 1024,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 1990,
      "existing_alias_text": "AI",
      "input_term": "AI",
      "matched_canonical": {
        "category_id": 2,
        "display_name": "AI",
        "id": 1347,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "CONCEPT",
        "slug": "ai",
        "sub_category_id": 1020,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    }
  ],
  "candidate_roles": [
    {
      "display_name": "Data Engineer",
      "id": 2,
      "rationale": null,
      "role_archetype": null,
      "slug": "data-engineer",
      "source": "db"
    },
    {
      "display_name": ".NET Backend Developer",
      "id": 83,
      "rationale": null,
      "role_archetype": "Engineering",
      "slug": "dotnet-backend-developer",
      "source": "db"
    },
    {
      "display_name": "Kotlin Backend Developer",
      "id": 84,
      "rationale": null,
      "role_archetype": "Engineering",
      "slug": "kotlin-server-backend-developer",
      "source": "db"
    },
    {
      "display_name": "Scala Backend Developer",
      "id": 87,
      "rationale": null,
      "role_archetype": "Engineering",
      "slug": "scala-backend-developer",
      "source": "db"
    },
    {
      "display_name": "Web Developer",
      "id": 25,
      "rationale": null,
      "role_archetype": null,
      "slug": "web-developer",
      "source": "db"
    },
    {
      "display_name": "Pega Developer",
      "id": 24,
      "rationale": null,
      "role_archetype": null,
      "slug": "pega-developer",
      "source": "db"
    },
    {
      "display_name": "ML Engineer",
      "id": 3,
      "rationale": null,
      "role_archetype": null,
      "slug": "ml-engineer",
      "source": "db"
    },
    {
      "display_name": "MLOps Engineer",
      "id": 16,
      "rationale": null,
      "role_archetype": null,
      "slug": "ml-ops-engineer",
      "source": "db"
    },
    {
      "display_name": "DevOps Engineer",
      "id": 10,
      "rationale": null,
      "role_archetype": null,
      "slug": "devops-engineer",
      "source": "db"
    },
    {
      "display_name": "AI Engineer",
      "id": 13,
      "rationale": null,
      "role_archetype": null,
      "slug": "ai-engineer",
      "source": "db"
    }
  ],
  "chosen_role": {
    "display_name": "Data Engineer",
    "id": 2,
    "rationale": "Domain=Data Engineering \u0026 Analytics; The JD centers on Databricks/Spark-based data migration, pipeline development, ETL modernization, optimization, and cloud data engineering, which best matches Data Engineer.",
    "role_archetype": null,
    "slug": "data-engineer",
    "source": "db"
  },
  "dimensions": [
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "React Frontend Development",
        "id": 96,
        "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
        "slug": "d_init_01",
        "source": "db"
      },
      "input_skill": "Databricks",
      "llm_role": null,
      "roles_from_db": []
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "ETL and ELT Tooling",
        "id": 24,
        "rationale": "Packaged tools for extracting, loading, and transforming data across systems. This dimension covers connector-based ingestion, transformation frameworks, and managed integration products.",
        "slug": "etl-and-elt-tooling",
        "source": "db"
      },
      "input_skill": "Apache Spark",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Data Engineer",
          "id": 2,
          "rationale": null,
          "role_archetype": null,
          "slug": "data-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Cloud Data Warehouses",
        "id": 22,
        "rationale": "Managed analytical storage and compute platforms used for curated datasets, reporting, and downstream analytics. These systems are central to data modeling, performance tuning, and cost-aware query design.",
        "slug": "cloud-data-warehouses",
        "source": "db"
      },
      "input_skill": "AWS Redshift",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Data Engineer",
          "id": 2,
          "rationale": null,
          "role_archetype": null,
          "slug": "data-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Cloud Platforms \u0026 Hosting Providers",
        "id": 278,
        "rationale": "Familiarity with vendor-specific hosting and backend services for deploying and scaling web applications.",
        "slug": "cloud-platforms-hosting-providers",
        "source": "db"
      },
      "input_skill": "Amazon RDS",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": ".NET Backend Developer",
          "id": 83,
          "rationale": null,
          "role_archetype": "Engineering",
          "slug": "dotnet-backend-developer",
          "source": "db"
        },
        {
          "display_name": "Kotlin Backend Developer",
          "id": 84,
          "rationale": null,
          "role_archetype": "Engineering",
          "slug": "kotlin-server-backend-developer",
          "source": "db"
        },
        {
          "display_name": "Scala Backend Developer",
          "id": 87,
          "rationale": null,
          "role_archetype": "Engineering",
          "slug": "scala-backend-developer",
          "source": "db"
        },
        {
          "display_name": "Web Developer",
          "id": 25,
          "rationale": null,
          "role_archetype": null,
          "slug": "web-developer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "React Frontend Development",
        "id": 96,
        "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
        "slug": "d_init_01",
        "source": "db"
      },
      "input_skill": "Lakehouse",
      "llm_role": null,
      "roles_from_db": []
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Pega Programming Languages \u0026 DSLs",
        "id": 267,
        "rationale": "Programming languages and domain-specific languages used in Pega development.",
        "slug": "pega-programming-languages-dsls",
        "source": "db"
      },
      "input_skill": "SQL",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Pega Developer",
          "id": 24,
          "rationale": null,
          "role_archetype": null,
          "slug": "pega-developer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Programming Languages for Data Work",
        "id": 21,
        "rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
        "slug": "programming-languages-for-data-work",
        "source": "db"
      },
      "input_skill": "SQL",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Data Engineer",
          "id": 2,
          "rationale": null,
          "role_archetype": null,
          "slug": "data-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "ETL and ELT Tooling",
        "id": 24,
        "rationale": "Packaged tools for extracting, loading, and transforming data across systems. This dimension covers connector-based ingestion, transformation frameworks, and managed integration products.",
        "slug": "etl-and-elt-tooling",
        "source": "db"
      },
      "input_skill": "PySpark",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Data Engineer",
          "id": 2,
          "rationale": null,
          "role_archetype": null,
          "slug": "data-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Model and Data Versioning",
        "id": 48,
        "rationale": "Versioning systems for datasets, features, and model artifacts at the storage layer. This enables reproducible training, rollback, lineage of artifacts, and controlled promotion of model assets.",
        "slug": "model-and-data-versioning",
        "source": "db"
      },
      "input_skill": "Delta Lake",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "ML Engineer",
          "id": 3,
          "rationale": null,
          "role_archetype": null,
          "slug": "ml-engineer",
          "source": "db"
        },
        {
          "display_name": "MLOps Engineer",
          "id": 16,
          "rationale": null,
          "role_archetype": null,
          "slug": "ml-ops-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "CI/CD Pipeline Platforms",
        "id": 150,
        "rationale": "Systems used to define, run, and maintain automated build and deployment workflows. This cluster is coherent because the role owns delivery automation end to end, including pipeline reliability and promotion logic.",
        "slug": "ci-cd-pipeline-platforms",
        "source": "db"
      },
      "input_skill": "CI/CD",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "DevOps Engineer",
          "id": 10,
          "rationale": null,
          "role_archetype": null,
          "slug": "devops-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "CI/CD for Machine Learning",
        "id": 56,
        "rationale": "Tools and platforms for automating ML model integration, testing, and deployment pipelines.",
        "slug": "ci-cd-for-machine-learning",
        "source": "db"
      },
      "input_skill": "CI/CD",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "ML Engineer",
          "id": 3,
          "rationale": null,
          "role_archetype": null,
          "slug": "ml-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Data Lineage and Metadata",
        "id": 28,
        "rationale": "Cataloging, documenting, and tracing how data moves and changes across systems. This dimension supports impact analysis, governance, discoverability, and operational understanding of datasets.",
        "slug": "data-lineage-and-metadata",
        "source": "db"
      },
      "input_skill": "Unity Catalog",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Data Engineer",
          "id": 2,
          "rationale": null,
          "role_archetype": null,
          "slug": "data-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "React Frontend Development",
        "id": 96,
        "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
        "slug": "d_init_01",
        "source": "db"
      },
      "input_skill": "Unity Catalog",
      "llm_role": null,
      "roles_from_db": []
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "AI Governance and Model Security",
        "id": 50,
        "rationale": "Controls and documentation used to make models safer, auditable, and compliant. ML engineers use this to manage model risk, supply chain integrity, and governance requirements.",
        "slug": "ai-governance-and-model-security",
        "source": "db"
      },
      "input_skill": "Machine Learning",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "AI Engineer",
          "id": 13,
          "rationale": null,
          "role_archetype": null,
          "slug": "ai-engineer",
          "source": "db"
        },
        {
          "display_name": "ML Engineer",
          "id": 3,
          "rationale": null,
          "role_archetype": null,
          "slug": "ml-engineer",
          "source": "db"
        },
        {
          "display_name": "MLOps Engineer",
          "id": 16,
          "rationale": null,
          "role_archetype": null,
          "slug": "ml-ops-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "React Frontend Development",
        "id": 96,
        "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
        "slug": "d_init_01",
        "source": "db"
      },
      "input_skill": "Machine Learning",
      "llm_role": null,
      "roles_from_db": []
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "React Frontend Development",
        "id": 96,
        "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
        "slug": "d_init_01",
        "source": "db"
      },
      "input_skill": "AI",
      "llm_role": null,
      "roles_from_db": []
    }
  ],
  "input_final_skills": [
    "Databricks",
    "Apache Spark",
    "AWS Redshift",
    "Amazon RDS",
    "Lakehouse",
    "ETL",
    "SQL",
    "PySpark",
    "Delta Lake",
    "ACID",
    "CI/CD",
    "Unity Catalog",
    "Machine Learning",
    "AI"
  ],
  "input_llm_skills": [
    "Databricks",
    "Apache Spark",
    "AWS Redshift",
    "Amazon RDS",
    "Lakehouse",
    "ETL",
    "SQL",
    "PySpark",
    "Delta Lake",
    "ACID",
    "CI/CD",
    "Unity Catalog",
    "Machine Learning",
    "AI"
  ],
  "new_aliases_persisted": 0,
  "run_id": "2ec7ccc9-2b00-4120-a2dd-aefabc025093",
  "skills_detail": [
    {
      "aliases_in_db": [
        {
          "alias_text": "Databricks",
          "alias_type": "CANONICAL",
          "id": 1838,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 9,
        "display_name": "Databricks",
        "id": 1202,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "PLATFORM",
        "slug": "databricks",
        "sub_category_id": 911,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "React Frontend Development",
            "id": 96,
            "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
            "slug": "d_init_01",
            "source": "db"
          },
          "input_skill": "Databricks",
          "llm_role": null,
          "roles_from_db": []
        }
      ],
      "input_skill": "Databricks",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "Apache Spark",
          "alias_type": "CANONICAL",
          "id": 2004,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "apache spark 3",
          "alias_type": "VERSION",
          "id": 2006,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "spark",
          "alias_type": "VERSION",
          "id": 2510,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "spark 3",
          "alias_type": "VERSION",
          "id": 2007,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "spark 3.x",
          "alias_type": "VERSION",
          "id": 2009,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "spark3",
          "alias_type": "VERSION",
          "id": 2008,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 5,
        "display_name": "Apache Spark",
        "id": 1350,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "FRAMEWORK",
        "slug": "apache-spark",
        "sub_category_id": 1021,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "ETL and ELT Tooling",
            "id": 24,
            "rationale": "Packaged tools for extracting, loading, and transforming data across systems. This dimension covers connector-based ingestion, transformation frameworks, and managed integration products.",
            "slug": "etl-and-elt-tooling",
            "source": "db"
          },
          "input_skill": "Apache Spark",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Data Engineer",
              "id": 2,
              "rationale": null,
              "role_archetype": null,
              "slug": "data-engineer",
              "source": "db"
            }
          ]
        }
      ],
      "input_skill": "Apache Spark",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "Amazon Redshift",
          "alias_type": "CANONICAL",
          "id": 301,
          "is_primary": true,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 11,
        "display_name": "Amazon Redshift",
        "id": 107,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "CLOUD_SERVICE",
        "slug": "amazon-redshift",
        "sub_category_id": 118,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Cloud Data Warehouses",
            "id": 22,
            "rationale": "Managed analytical storage and compute platforms used for curated datasets, reporting, and downstream analytics. These systems are central to data modeling, performance tuning, and cost-aware query design.",
            "slug": "cloud-data-warehouses",
            "source": "db"
          },
          "input_skill": "AWS Redshift",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Data Engineer",
              "id": 2,
              "rationale": null,
              "role_archetype": null,
              "slug": "data-engineer",
              "source": "db"
            }
          ]
        }
      ],
      "input_skill": "AWS Redshift",
      "matched_via": "embedding_alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "AWS RDS",
          "alias_type": "CANONICAL",
          "id": 5229,
          "is_primary": true,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 11,
        "display_name": "AWS RDS",
        "id": 3632,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "CLOUD_SERVICE",
        "slug": "aws-rds",
        "sub_category_id": 2845,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Cloud Platforms \u0026 Hosting Providers",
            "id": 278,
            "rationale": "Familiarity with vendor-specific hosting and backend services for deploying and scaling web applications.",
            "slug": "cloud-platforms-hosting-providers",
            "source": "db"
          },
          "input_skill": "Amazon RDS",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": ".NET Backend Developer",
              "id": 83,
              "rationale": null,
              "role_archetype": "Engineering",
              "slug": "dotnet-backend-developer",
              "source": "db"
            },
            {
              "display_name": "Kotlin Backend Developer",
              "id": 84,
              "rationale": null,
              "role_archetype": "Engineering",
              "slug": "kotlin-server-backend-developer",
              "source": "db"
            },
            {
              "display_name": "Scala Backend Developer",
              "id": 87,
              "rationale": null,
              "role_archetype": "Engineering",
              "slug": "scala-backend-developer",
              "source": "db"
            },
            {
              "display_name": "Web Developer",
              "id": 25,
              "rationale": null,
              "role_archetype": null,
              "slug": "web-developer",
              "source": "db"
            }
          ]
        }
      ],
      "input_skill": "Amazon RDS",
      "matched_via": "embedding_alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "Lakehouse",
          "alias_type": "CANONICAL",
          "id": 2018,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 1,
        "display_name": "Lakehouse",
        "id": 1359,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "PATTERN",
        "slug": "lakehouse",
        "sub_category_id": 1026,
        "typical_lifespan": "EVERGREEN",
        "volatility": "EMERGING"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "React Frontend Development",
            "id": 96,
            "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
            "slug": "d_init_01",
            "source": "db"
          },
          "input_skill": "Lakehouse",
          "llm_role": null,
          "roles_from_db": []
        }
      ],
      "input_skill": "Lakehouse",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [],
      "canonical": null,
      "dimensions": [],
      "input_skill": "ETL",
      "matched_via": null,
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": {
        "derived": {
          "category": "Data Engineering Tools",
          "skill_nature": "PRACTICE",
          "sub_category": "general",
          "typical_lifespan": "MULTI_YEAR",
          "version_strategy": "UNVERSIONED",
          "volatility": "MEDIUM"
        },
        "enrichment": null,
        "keep_log": [],
        "locked_dimensions": [],
        "merge_log": [],
        "placed": null,
        "relationships": null,
        "skill_id": "etl",
        "split_log": [],
        "typed": null,
        "warnings": []
      },
      "source_tag": "llm",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "SQL",
          "alias_type": "CANONICAL",
          "id": 271,
          "is_primary": true,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 6,
        "display_name": "SQL",
        "id": 101,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "LANGUAGE",
        "slug": "sql",
        "sub_category_id": 97,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Pega Programming Languages \u0026 DSLs",
            "id": 267,
            "rationale": "Programming languages and domain-specific languages used in Pega development.",
            "slug": "pega-programming-languages-dsls",
            "source": "db"
          },
          "input_skill": "SQL",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Pega Developer",
              "id": 24,
              "rationale": null,
              "role_archetype": null,
              "slug": "pega-developer",
              "source": "db"
            }
          ]
        },
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Programming Languages for Data Work",
            "id": 21,
            "rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
            "slug": "programming-languages-for-data-work",
            "source": "db"
          },
          "input_skill": "SQL",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Data Engineer",
              "id": 2,
              "rationale": null,
              "role_archetype": null,
              "slug": "data-engineer",
              "source": "db"
            }
          ]
        }
      ],
      "input_skill": "SQL",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "Apache Spark",
          "alias_type": "CANONICAL",
          "id": 2004,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "apache spark 3",
          "alias_type": "VERSION",
          "id": 2006,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "spark",
          "alias_type": "VERSION",
          "id": 2510,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "spark 3",
          "alias_type": "VERSION",
          "id": 2007,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "spark 3.x",
          "alias_type": "VERSION",
          "id": 2009,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "spark3",
          "alias_type": "VERSION",
          "id": 2008,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 5,
        "display_name": "Apache Spark",
        "id": 1350,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "FRAMEWORK",
        "slug": "apache-spark",
        "sub_category_id": 1021,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "ETL and ELT Tooling",
            "id": 24,
            "rationale": "Packaged tools for extracting, loading, and transforming data across systems. This dimension covers connector-based ingestion, transformation frameworks, and managed integration products.",
            "slug": "etl-and-elt-tooling",
            "source": "db"
          },
          "input_skill": "PySpark",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Data Engineer",
              "id": 2,
              "rationale": null,
              "role_archetype": null,
              "slug": "data-engineer",
              "source": "db"
            }
          ]
        }
      ],
      "input_skill": "PySpark",
      "matched_via": "embedding_alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "Delta Lake",
          "alias_type": "CANONICAL",
          "id": 498,
          "is_primary": true,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 13,
        "display_name": "Delta Lake",
        "id": 237,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "TOOL",
        "slug": "delta-lake",
        "sub_category_id": 1170,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Model and Data Versioning",
            "id": 48,
            "rationale": "Versioning systems for datasets, features, and model artifacts at the storage layer. This enables reproducible training, rollback, lineage of artifacts, and controlled promotion of model assets.",
            "slug": "model-and-data-versioning",
            "source": "db"
          },
          "input_skill": "Delta Lake",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "ML Engineer",
              "id": 3,
              "rationale": null,
              "role_archetype": null,
              "slug": "ml-engineer",
              "source": "db"
            },
            {
              "display_name": "MLOps Engineer",
              "id": 16,
              "rationale": null,
              "role_archetype": null,
              "slug": "ml-ops-engineer",
              "source": "db"
            }
          ]
        }
      ],
      "input_skill": "Delta Lake",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [],
      "canonical": null,
      "dimensions": [],
      "input_skill": "ACID",
      "matched_via": null,
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": {
        "derived": {
          "category": "Concepts",
          "skill_nature": "CONCEPT",
          "sub_category": "general",
          "typical_lifespan": "EVERGREEN",
          "version_strategy": "UNVERSIONED",
          "volatility": "STABLE"
        },
        "enrichment": null,
        "keep_log": [],
        "locked_dimensions": [],
        "merge_log": [],
        "placed": null,
        "relationships": null,
        "skill_id": "acid",
        "split_log": [],
        "typed": null,
        "warnings": []
      },
      "source_tag": "llm",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "CI/CD",
          "alias_type": "CANONICAL",
          "id": 1826,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 8,
        "display_name": "CI/CD",
        "id": 1190,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "METHODOLOGY",
        "slug": "ci-cd",
        "sub_category_id": 900,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "CI/CD Pipeline Platforms",
            "id": 150,
            "rationale": "Systems used to define, run, and maintain automated build and deployment workflows. This cluster is coherent because the role owns delivery automation end to end, including pipeline reliability and promotion logic.",
            "slug": "ci-cd-pipeline-platforms",
            "source": "db"
          },
          "input_skill": "CI/CD",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "DevOps Engineer",
              "id": 10,
              "rationale": null,
              "role_archetype": null,
              "slug": "devops-engineer",
              "source": "db"
            }
          ]
        },
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "CI/CD for Machine Learning",
            "id": 56,
            "rationale": "Tools and platforms for automating ML model integration, testing, and deployment pipelines.",
            "slug": "ci-cd-for-machine-learning",
            "source": "db"
          },
          "input_skill": "CI/CD",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "ML Engineer",
              "id": 3,
              "rationale": null,
              "role_archetype": null,
              "slug": "ml-engineer",
              "source": "db"
            }
          ]
        }
      ],
      "input_skill": "CI/CD",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "Unity Catalog",
          "alias_type": "CANONICAL",
          "id": 1839,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 9,
        "display_name": "Unity Catalog",
        "id": 1203,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "PLATFORM",
        "slug": "unity-catalog",
        "sub_category_id": 250,
        "typical_lifespan": "EVERGREEN",
        "volatility": "EMERGING"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Data Lineage and Metadata",
            "id": 28,
            "rationale": "Cataloging, documenting, and tracing how data moves and changes across systems. This dimension supports impact analysis, governance, discoverability, and operational understanding of datasets.",
            "slug": "data-lineage-and-metadata",
            "source": "db"
          },
          "input_skill": "Unity Catalog",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Data Engineer",
              "id": 2,
              "rationale": null,
              "role_archetype": null,
              "slug": "data-engineer",
              "source": "db"
            }
          ]
        },
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "React Frontend Development",
            "id": 96,
            "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
            "slug": "d_init_01",
            "source": "db"
          },
          "input_skill": "Unity Catalog",
          "llm_role": null,
          "roles_from_db": []
        }
      ],
      "input_skill": "Unity Catalog",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "Machine Learning",
          "alias_type": "CANONICAL",
          "id": 2015,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 2,
        "display_name": "Machine Learning",
        "id": 1356,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "CONCEPT",
        "slug": "machine-learning",
        "sub_category_id": 1024,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "AI Governance and Model Security",
            "id": 50,
            "rationale": "Controls and documentation used to make models safer, auditable, and compliant. ML engineers use this to manage model risk, supply chain integrity, and governance requirements.",
            "slug": "ai-governance-and-model-security",
            "source": "db"
          },
          "input_skill": "Machine Learning",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "AI Engineer",
              "id": 13,
              "rationale": null,
              "role_archetype": null,
              "slug": "ai-engineer",
              "source": "db"
            },
            {
              "display_name": "ML Engineer",
              "id": 3,
              "rationale": null,
              "role_archetype": null,
              "slug": "ml-engineer",
              "source": "db"
            },
            {
              "display_name": "MLOps Engineer",
              "id": 16,
              "rationale": null,
              "role_archetype": null,
              "slug": "ml-ops-engineer",
              "source": "db"
            }
          ]
        },
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "React Frontend Development",
            "id": 96,
            "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
            "slug": "d_init_01",
            "source": "db"
          },
          "input_skill": "Machine Learning",
          "llm_role": null,
          "roles_from_db": []
        }
      ],
      "input_skill": "Machine Learning",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "AI",
          "alias_type": "CANONICAL",
          "id": 1990,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 2,
        "display_name": "AI",
        "id": 1347,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "CONCEPT",
        "slug": "ai",
        "sub_category_id": 1020,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "React Frontend Development",
            "id": 96,
            "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
            "slug": "d_init_01",
            "source": "db"
          },
          "input_skill": "AI",
          "llm_role": null,
          "roles_from_db": []
        }
      ],
      "input_skill": "AI",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    }
  ],
  "unmatched_skills": [
    "ETL",
    "ACID"
  ]
}
API 3 — final-role-output
{
  "chosen_role": {
    "display_name": "Data Engineer",
    "id": 2,
    "rationale": "Domain=Data Engineering \u0026 Analytics; The JD centers on Databricks/Spark-based data migration, pipeline development, ETL modernization, optimization, and cloud data engineering, which best matches Data Engineer.",
    "role_archetype": null,
    "slug": "data-engineer",
    "source": "db"
  },
  "chosen_role_resolution": "in_db",
  "final_input_skills": [
    {
      "skill": "Databricks",
      "tag": "in_db"
    },
    {
      "skill": "Apache Spark",
      "tag": "in_db"
    },
    {
      "skill": "AWS Redshift",
      "tag": "in_db"
    },
    {
      "skill": "Amazon RDS",
      "tag": "in_db"
    },
    {
      "skill": "Lakehouse",
      "tag": "in_db"
    },
    {
      "skill": "ETL",
      "tag": "new"
    },
    {
      "skill": "SQL",
      "tag": "in_db"
    },
    {
      "skill": "PySpark",
      "tag": "in_db"
    },
    {
      "skill": "Delta Lake",
      "tag": "in_db"
    },
    {
      "skill": "ACID",
      "tag": "new"
    },
    {
      "skill": "CI/CD",
      "tag": "in_db"
    },
    {
      "skill": "Unity Catalog",
      "tag": "in_db"
    },
    {
      "skill": "Machine Learning",
      "tag": "in_db"
    },
    {
      "skill": "AI",
      "tag": "in_db"
    }
  ],
  "llm_cost_api1_usd": null,
  "llm_cost_api2_usd": null,
  "llm_cost_api3_usd": null,
  "llm_cost_total_usd": null,
  "persistence": {
    "items": [
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "React Frontend Development",
          "id": 96,
          "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
          "slug": "d_init_01",
          "source": "db"
        },
        "dimension_id": 96,
        "input_skill": "Databricks",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [],
        "skill_dimension_saved": true,
        "skill_id": 1202,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "ETL and ELT Tooling",
          "id": 24,
          "rationale": "Packaged tools for extracting, loading, and transforming data across systems. This dimension covers connector-based ingestion, transformation frameworks, and managed integration products.",
          "slug": "etl-and-elt-tooling",
          "source": "db"
        },
        "dimension_id": 24,
        "input_skill": "Apache Spark",
        "llm_role": null,
        "matched_chosen_role": true,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
        "role_dimension_saved": true,
        "roles_from_db": [
          {
            "display_name": "Data Engineer",
            "id": 2,
            "rationale": null,
            "role_archetype": null,
            "slug": "data-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 1350,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Cloud Data Warehouses",
          "id": 22,
          "rationale": "Managed analytical storage and compute platforms used for curated datasets, reporting, and downstream analytics. These systems are central to data modeling, performance tuning, and cost-aware query design.",
          "slug": "cloud-data-warehouses",
          "source": "db"
        },
        "dimension_id": 22,
        "input_skill": "AWS Redshift",
        "llm_role": null,
        "matched_chosen_role": true,
        "outcome_line": "Skipped \u2014 no persistable v3 meta for new skill",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "Data Engineer",
            "id": 2,
            "rationale": null,
            "role_archetype": null,
            "slug": "data-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": false,
        "skill_id": null,
        "skill_tag": "new",
        "skipped_reason": "skill_not_in_db_v3_proposed"
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Cloud Platforms \u0026 Hosting Providers",
          "id": 278,
          "rationale": "Familiarity with vendor-specific hosting and backend services for deploying and scaling web applications.",
          "slug": "cloud-platforms-hosting-providers",
          "source": "db"
        },
        "dimension_id": 278,
        "input_skill": "Amazon RDS",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Skipped \u2014 no persistable v3 meta for new skill",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": ".NET Backend Developer",
            "id": 83,
            "rationale": null,
            "role_archetype": "Engineering",
            "slug": "dotnet-backend-developer",
            "source": "db"
          },
          {
            "display_name": "Kotlin Backend Developer",
            "id": 84,
            "rationale": null,
            "role_archetype": "Engineering",
            "slug": "kotlin-server-backend-developer",
            "source": "db"
          },
          {
            "display_name": "Scala Backend Developer",
            "id": 87,
            "rationale": null,
            "role_archetype": "Engineering",
            "slug": "scala-backend-developer",
            "source": "db"
          },
          {
            "display_name": "Web Developer",
            "id": 25,
            "rationale": null,
            "role_archetype": null,
            "slug": "web-developer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": false,
        "skill_id": null,
        "skill_tag": "new",
        "skipped_reason": "skill_not_in_db_v3_proposed"
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "React Frontend Development",
          "id": 96,
          "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
          "slug": "d_init_01",
          "source": "db"
        },
        "dimension_id": 96,
        "input_skill": "Lakehouse",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [],
        "skill_dimension_saved": true,
        "skill_id": 1359,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Pega Programming Languages \u0026 DSLs",
          "id": 267,
          "rationale": "Programming languages and domain-specific languages used in Pega development.",
          "slug": "pega-programming-languages-dsls",
          "source": "db"
        },
        "dimension_id": 267,
        "input_skill": "SQL",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "Pega Developer",
            "id": 24,
            "rationale": null,
            "role_archetype": null,
            "slug": "pega-developer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 101,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Programming Languages for Data Work",
          "id": 21,
          "rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
          "slug": "programming-languages-for-data-work",
          "source": "db"
        },
        "dimension_id": 21,
        "input_skill": "SQL",
        "llm_role": null,
        "matched_chosen_role": true,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
        "role_dimension_saved": true,
        "roles_from_db": [
          {
            "display_name": "Data Engineer",
            "id": 2,
            "rationale": null,
            "role_archetype": null,
            "slug": "data-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 101,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "ETL and ELT Tooling",
          "id": 24,
          "rationale": "Packaged tools for extracting, loading, and transforming data across systems. This dimension covers connector-based ingestion, transformation frameworks, and managed integration products.",
          "slug": "etl-and-elt-tooling",
          "source": "db"
        },
        "dimension_id": 24,
        "input_skill": "PySpark",
        "llm_role": null,
        "matched_chosen_role": true,
        "outcome_line": "Skipped \u2014 no persistable v3 meta for new skill",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "Data Engineer",
            "id": 2,
            "rationale": null,
            "role_archetype": null,
            "slug": "data-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": false,
        "skill_id": null,
        "skill_tag": "new",
        "skipped_reason": "skill_not_in_db_v3_proposed"
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Model and Data Versioning",
          "id": 48,
          "rationale": "Versioning systems for datasets, features, and model artifacts at the storage layer. This enables reproducible training, rollback, lineage of artifacts, and controlled promotion of model assets.",
          "slug": "model-and-data-versioning",
          "source": "db"
        },
        "dimension_id": 48,
        "input_skill": "Delta Lake",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "ML Engineer",
            "id": 3,
            "rationale": null,
            "role_archetype": null,
            "slug": "ml-engineer",
            "source": "db"
          },
          {
            "display_name": "MLOps Engineer",
            "id": 16,
            "rationale": null,
            "role_archetype": null,
            "slug": "ml-ops-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 237,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "CI/CD Pipeline Platforms",
          "id": 150,
          "rationale": "Systems used to define, run, and maintain automated build and deployment workflows. This cluster is coherent because the role owns delivery automation end to end, including pipeline reliability and promotion logic.",
          "slug": "ci-cd-pipeline-platforms",
          "source": "db"
        },
        "dimension_id": 150,
        "input_skill": "CI/CD",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "DevOps Engineer",
            "id": 10,
            "rationale": null,
            "role_archetype": null,
            "slug": "devops-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 1190,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "CI/CD for Machine Learning",
          "id": 56,
          "rationale": "Tools and platforms for automating ML model integration, testing, and deployment pipelines.",
          "slug": "ci-cd-for-machine-learning",
          "source": "db"
        },
        "dimension_id": 56,
        "input_skill": "CI/CD",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "ML Engineer",
            "id": 3,
            "rationale": null,
            "role_archetype": null,
            "slug": "ml-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 1190,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Data Lineage and Metadata",
          "id": 28,
          "rationale": "Cataloging, documenting, and tracing how data moves and changes across systems. This dimension supports impact analysis, governance, discoverability, and operational understanding of datasets.",
          "slug": "data-lineage-and-metadata",
          "source": "db"
        },
        "dimension_id": 28,
        "input_skill": "Unity Catalog",
        "llm_role": null,
        "matched_chosen_role": true,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
        "role_dimension_saved": true,
        "roles_from_db": [
          {
            "display_name": "Data Engineer",
            "id": 2,
            "rationale": null,
            "role_archetype": null,
            "slug": "data-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 1203,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "React Frontend Development",
          "id": 96,
          "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
          "slug": "d_init_01",
          "source": "db"
        },
        "dimension_id": 96,
        "input_skill": "Unity Catalog",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [],
        "skill_dimension_saved": true,
        "skill_id": 1203,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "AI Governance and Model Security",
          "id": 50,
          "rationale": "Controls and documentation used to make models safer, auditable, and compliant. ML engineers use this to manage model risk, supply chain integrity, and governance requirements.",
          "slug": "ai-governance-and-model-security",
          "source": "db"
        },
        "dimension_id": 50,
        "input_skill": "Machine Learning",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "AI Engineer",
            "id": 13,
            "rationale": null,
            "role_archetype": null,
            "slug": "ai-engineer",
            "source": "db"
          },
          {
            "display_name": "ML Engineer",
            "id": 3,
            "rationale": null,
            "role_archetype": null,
            "slug": "ml-engineer",
            "source": "db"
          },
          {
            "display_name": "MLOps Engineer",
            "id": 16,
            "rationale": null,
            "role_archetype": null,
            "slug": "ml-ops-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 1356,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "React Frontend Development",
          "id": 96,
          "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
          "slug": "d_init_01",
          "source": "db"
        },
        "dimension_id": 96,
        "input_skill": "Machine Learning",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [],
        "skill_dimension_saved": true,
        "skill_id": 1356,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "React Frontend Development",
          "id": 96,
          "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
          "slug": "d_init_01",
          "source": "db"
        },
        "dimension_id": 96,
        "input_skill": "AI",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [],
        "skill_dimension_saved": true,
        "skill_id": 1347,
        "skill_tag": "in_db",
        "skipped_reason": null
      }
    ],
    "new_skills_created": 0,
    "role_dimension_saved": 0,
    "skill_dimension_saved": 0,
    "skipped": 3
  },
  "planner_output": null,
  "run_id": "2ec7ccc9-2b00-4120-a2dd-aefabc025093"
}

LLM Calls

Every model call made for this run, in pipeline order. Click a card to see the model's response.

Loading…