← Back to history

Pipeline run

459d8d5b-8e49-4c9b-a801-9389a9c4c4ae

Pipeline LLM cost (USD)
API 1: $0.0048 API 2: $0.0005 API 3: $0.0000 Total: $0.0054

Client output enrichment

v2 Skill cluster · Nature of work · AI index · Tech stack maturity · Evidence · KRA description
role baseline loaded sources · ai_index: jd · nature_of_work: jd · tech_stack_maturity: jd
Nature of work · Data platform optimization and governance
Lead the design and rollout of a global data platform: set data engineering roadmap/architecture, build and optimize large-scale pipelines and datasets, document and train teams, and partner with product/engineering leadership on measurement and strategy.
"own and drive strategic planning, business analysis and technical implementation of data engineering, tools, and data architecture definition"
Tech stack maturity
Mainstream Legacy
Apache Spark, Hadoop, Hive, NoSQL, Python, Scala, and SQL together indicate a mature big-data stack centered on established distributed processing and warehouse technologies rather than cutting-edge cloud-native platforms.
AI index (0 = no AI use, 5 = totally AI-dependent · v2.1)
0.50 / 5
· Title match
Has AI skill
· AI skill (primary)
AI skill (secondary)
· On AI team
· Builds AI products
vocab breakdown (legacy)
Assistants (×1):
Frameworks (×2):
Models / concepts (×3): Machine Learning
Evidence — skills matched in JD (31)
Data Engineering Data Architecture SQL Python Apache Spark Hive Scala Hadoop EMR NoSQL Unix Shell Oozie Airflow Apache Storm Spark Streaming NiFi Tableau Power BI D3 Git CI/CD Master Data Management Machine Learning SOA +6
Skill cluster (6 dimension groups, role-scoped)
Programming Languages for Data Work
SQL Python Scala
BI and Visualization Tools
Tableau Power BI
ETL and ELT Tooling
Apache Spark Hadoop
AI Governance and Model Security
Machine Learning
Stream Processing Systems
Spark Streaming
Cross-cutting / unaligned
Data Engineering Data Architecture Hive EMR NoSQL Unix Shell Oozie Airflow Apache Storm NiFi D3 Git CI/CD Master Data Management SOA API Development Agile Scrum MLlib GraphX Pandas
Show KRA description ↓
As Lead Data Engineer you will design and implement the next version of our integrated data platform.You will own and drive strategic planning, business analysis and technical implementation of data engineering, tools, and data architecture definition.In addition to managing our portfolio of datasets globally, you will play key role in leading our data engineering community to leverage these datasets effectively.You will also produce documentation and lead training sessions for data engineering solutions.Collaborate with and influence leadership so that your teams work directly impacts company strategy and direction.Develop new techniques and data that will enable answering previously unanswerable and new questions.Develop deep partnerships with engineering and product teams to deliver on major cross-functional measurements, testing.Communicate effectively to all levels of the organization, including executives.Provide success in partnering teams with dramatically varying backgrounds, from the highly technical to the highly creative. Experience designing a data engineering roadmap and executing the vision behind it.Experience instilling a culture of ownership, collaboration, and results.Track record of recruiting talent in data engineering, analytics, and data science.Good business acumen to orient data analysis to business needs of clients.Ability to translate business concepts into a technical architectureAbility to collaborate with cross-functional teams to define priorities and develop roadmapsProject management skillsDemonstrated intellectual and analytical rigor, strong attention to detail Team oriented, energetic, collaborative, diplomatic, and flexible styleYou will have the opportunity to be very autonomous, self-driven, able to navigate and lead through ambiguityYou excel at articulating complex topics to the layman (understands audience and speaks to be understood)Experience with crafting and operating scalable, resilient, highly available servicesYou have a deep understanding and appreciation of both machine-readable standards and master data management (MDM)Work experience in an analytical field such as computer science.5+ years of experience in Data EngineeringWorking knowledge of Hadoop ecosystem and associated technologies, (For e. g., Apache Spark, MLlib, GraphX, iPython, sci-kit, Pandas etc. )Experience in creating large scale data engineering pipelines, data-based decision-making, and quantitative analysis.Exposure with Visualization Tools like Tableau, Power BI, D3 and exposure to code version control systems (git).Experience - CICD/ Data Architectures/ Pipelines/ Quality /Code Management.Advanced experience in writing and optimizing efficient SQL queries with Python, Spark, Hive, Scala handling Large Data Sets in Big-Data Environments.Experience with SQL for extracting, aggregating, and processing big data Pipelines using Hadoop, EMR & NoSQL Databases.Experience creating/supporting production software/systems and a proven track record of identifying and resolving performance bottlenecks for production systems.Experience with Unix/Shell and exposure to Scheduling tools like Oozie and Airflow.Exposure to stream-processing systems like Apache Storm, Spark-Streaming.Experience with Big Data platforms such as Apache Hadoop and Apache Spark, NiFiUnderstanding of Machine Learning technology concepts and applicationExperience implementing SOA frameworkExperience with enterprise library development and management Experience with API development and managementExperience working in agile scrum environmentAbility to translate data and technical concepts into requirements documents, business cases.

Signals

Skill data-engineer
0.42
Alias data-engineer
1.00
KRA data-engineer
0.56

Post-classification

Centroidupdated · n=197
Alias collision log
New-role queue
New skills captured15
New KRA captured

Captured for admin review

Data Engineering primary Data Engineer pending
Data Architecture primary Data Engineer pending
EMR primary Data Engineer pending
Unix primary Data Engineer pending
Shell primary Data Engineer pending
Oozie Data Engineer pending
Apache Storm Data Engineer pending
NiFi Data Engineer pending
D3 Data Engineer pending
Master Data Management Data Engineer pending
SOA Data Engineer pending
API Development Data Engineer pending
MLlib Data Engineer pending
GraphX Data Engineer pending
Pandas Data Engineer pending
Status: completed Created: 2026-05-27T14:40:25.020682Z Updated: 2026-06-12T17:23:14.585728Z API 3 duration: 47500 ms
Flow Current 3-step pipeline

1 POST /skills/extract-from-jd

2 POST /skills/extract-details

3 POST /skills/final-role-output

Role Chosen role & resolution

Data Engineer

CASE A

slug: data-engineer · id: 2 · source: db

Exact alias hit on data-engineer (1.0) — no other alias at this confidence; skill_top data-engineer 0.42 does not contradict

Resolution: in_db — role exists in library; skill↔dim and role↔dim links saved when applicable.

0
New skills
0
Skill↔dim saved
0
Role↔dim saved
0
Skipped

Job description

Responsibilities
As Lead Data Engineer you will design and implement the next version of our integrated data platform.You will own and drive strategic planning, business analysis and technical implementation of data engineering, tools, and data architecture definition.In addition to managing our portfolio of datasets globally, you will play key role in leading our data engineering community to leverage these datasets effectively.You will also produce documentation and lead training sessions for data engineering solutions.Collaborate with and influence leadership so that your teams work directly impacts company strategy and direction.Develop new techniques and data that will enable answering previously unanswerable and new questions.Develop deep partnerships with engineering and product teams to deliver on major cross-functional measurements, testing.Communicate effectively to all levels of the organization, including executives.Provide success in partnering teams with dramatically varying backgrounds, from the highly technical to the highly creative.
Requirements
Experience designing a data engineering roadmap and executing the vision behind it.Experience instilling a culture of ownership, collaboration, and results.Track record of recruiting talent in data engineering, analytics, and data science.Good business acumen to orient data analysis to business needs of clients.Ability to translate business concepts into a technical architectureAbility to collaborate with cross-functional teams to define priorities and develop roadmapsProject management skillsDemonstrated intellectual and analytical rigor, strong attention to detail Team oriented, energetic, collaborative, diplomatic, and flexible styleYou will have the opportunity to be very autonomous, self-driven, able to navigate and lead through ambiguityYou excel at articulating complex topics to the layman (understands audience and speaks to be understood)Experience with crafting and operating scalable, resilient, highly available servicesYou have a deep understanding and appreciation of both machine-readable standards and master data management (MDM)Work experience in an analytical field such as computer science.5+ years of experience in Data EngineeringWorking knowledge of Hadoop ecosystem and associated technologies, (For e. g., Apache Spark, MLlib, GraphX, iPython, sci-kit, Pandas etc. )Experience in creating large scale data engineering pipelines, data-based decision-making, and quantitative analysis.Exposure with Visualization Tools like Tableau, Power BI, D3 and exposure to code version control systems (git).Experience - CICD/ Data Architectures/ Pipelines/ Quality /Code Management.Advanced experience in writing and optimizing efficient SQL queries with Python, Spark, Hive, Scala handling Large Data Sets in Big-Data Environments.Experience with SQL for extracting, aggregating, and processing big data Pipelines using Hadoop, EMR & NoSQL Databases.Experience creating/supporting production software/systems and a proven track record of identifying and resolving performance bottlenecks for production systems.Experience with Unix/Shell and exposure to Scheduling tools like Oozie and Airflow.Exposure to stream-processing systems like Apache Storm, Spark-Streaming.Experience with Big Data platforms such as Apache Hadoop and Apache Spark, NiFiUnderstanding of Machine Learning technology concepts and applicationExperience implementing SOA frameworkExperience with enterprise library development and management Experience with API development and managementExperience working in agile scrum environmentAbility to translate data and technical concepts into requirements documents, business cases.

Skills from this JD

Each row merges API 1 extraction, API 2 library match / v3 orchestration (dimensions + locked dims), and API 3 persistence tags.

Data Engineering Primary New / orchestrated API 3: new canonical path (new) New / unmatched skill (orchestrated in API 2)

Skill enrichment (orchestrator / LLM)

No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).

Derived legacy fields
Category
Data Engineering Tools
Sub-category
general
Skill nature
PRACTICE
Volatility
MEDIUM
Typical lifespan
MULTI_YEAR
Version strategy
UNVERSIONED
Data Architecture Primary New / orchestrated API 3: new canonical path (new) New / unmatched skill (orchestrated in API 2)

Skill enrichment (orchestrator / LLM)

No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).

Derived legacy fields
Category
Data Engineering Tools
Sub-category
general
Skill nature
CONCEPT
Volatility
MEDIUM
Typical lifespan
MULTI_YEAR
Version strategy
UNVERSIONED
SQL Primary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: SQL id=101 · sql

Aliases — catalog

  • SQL (CANONICAL) primary

Context tags (catalog)

ACID CTE DDL DML ETL JOIN MySQL NoSQL OLAP ORM PostgreSQL SQL injection SQLite T-SQL data modeling data warehousing database normalization execution plan indexing joins normalization query optimization stored procedures subquery transaction isolation transaction management window functions

Stored enrichment (catalog DB)

Category
Language
Sub-category
Query Language
Vendor
ANSI
License
unknown
Year introduced
1974
Confidence
0.99
Version strategy
NOT_APPLICABLE

Maturity reasoning: SQL appears in a large share of data, backend, and analytics job descriptions and remains the default query language for PostgreSQL, MySQL, and cloud warehouses like Snowflake/BigQuery.

Skill profile (library / DB)

Skill nature
LANGUAGE
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
6
Sub-category id
97
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • Pega Programming Languages & DSLs Catalog dimension db id 267

    Library dimension (catalog)

    Roles linked in library: Pega Developer

  • Programming Languages for Data Work Catalog dimension db id 21

    Library dimension (catalog)

    Roles linked in library: Data Engineer

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
Pega Programming Languages & DSLs
pega-programming-languages-dsls
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Programming Languages for Data Work
programming-languages-for-data-work
Existing dimension (library) · Role↔dimension saved
Python Primary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Python id=5 · python

Aliases — catalog

  • Python (CANONICAL) primary
  • Python 2 (VERSION)
  • Python 2.x (VERSION)
  • Python 3 (VERSION)
  • Python 3.10 (VERSION)
  • Python 3.11 (VERSION)
  • Python 3.12 (VERSION)
  • Python 3.x (VERSION)
  • py (VERSION)
  • py2 (VERSION)
  • py3 (VERSION)
  • python 3 (VERSION)
  • python 3.x (VERSION)
  • python2 (VERSION)
  • python3 (VERSION)
  • python3.x (VERSION)

Context tags (catalog)

API Django FastAPI Flask Jupyter NumPy PEP 8 Pandas REST SQLAlchemy asyncio pandas pip pytest type hints venv virtualenv

Stored enrichment (catalog DB)

Category
Language
Sub-category
Programming Language
Vendor
PSF
License
mit
Year introduced
1991
Confidence
0.99
Version strategy
SEPARATE_ENTITY
Version tag
3

Maturity reasoning: Python appears in a very high volume of job descriptions across data, backend, automation, and ML roles, and remains a default hiring-pipeline language on major job boards and tech stacks.

Skill profile (library / DB)

Skill nature
LANGUAGE
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
6
Sub-category id
96
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • Cloud Security Scripting & DSL Languages Catalog dimension db id 248

    Library dimension (catalog)

    Roles linked in library: Cloud Security Engineer

  • Programming Languages Catalog dimension db id 1

    Library dimension (catalog)

    Roles linked in library: Backend Developer, Fullstack Developer, Fullstack Developer

  • Programming Languages and Scripting Catalog dimension db id 59

    Library dimension (catalog)

    Roles linked in library: Cyber Security Engineer

  • Programming Languages for Data Work Catalog dimension db id 21

    Library dimension (catalog)

    Roles linked in library: Data Engineer

  • Programming Languages for ML Systems Catalog dimension db id 39

    Library dimension (catalog)

    Roles linked in library: ML Engineer, MLOps Engineer

  • Programming Languages for XR Catalog dimension db id 97

    Library dimension (catalog)

    Roles linked in library: AR/VR Engineer

  • Python Programming Catalog dimension db id 290

    Library dimension (catalog)

    Roles linked in library: Python Backend Developer

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
Cloud Security Scripting & DSL Languages
cloud-security-scripting-dsl-languages
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Programming Languages
programming-languages
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Programming Languages and Scripting
programming-languages-and-scripting
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Programming Languages for Data Work
programming-languages-for-data-work
Existing dimension (library) · Role↔dimension saved
Programming Languages for ML Systems
programming-languages-for-ml-systems
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Programming Languages for XR
programming-languages-for-xr
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Python Programming
python-programming
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Apache Spark Primary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Apache Spark id=1350 · apache-spark

Aliases — catalog

  • Apache Spark (CANONICAL)
  • apache spark 3 (VERSION)
  • spark (VERSION)
  • spark 3 (VERSION)
  • spark 3.x (VERSION)
  • spark3 (VERSION)

Context tags (catalog)

Apache Kafka Cluster Manager DAGScheduler Data Lake DataFrame ETL Hadoop MLlib Machine Learning PySpark RDD Scala Spark SQL Spark Streaming SparkSession

Stored enrichment (catalog DB)

Category
Framework
Sub-category
Distributed Data Processing Framework
Vendor
Apache Software Foundation
License
apache_2
Year introduced
2010
Confidence
0.94
Version strategy
SEPARATE_ENTITY
Version tag
3.x

Maturity reasoning: Apache Spark appears in many data engineering JDs and remains a standard for distributed ETL/ELT; its GitHub and vendor ecosystem activity stay strong, with Databricks and cloud platforms still promoting it.

Skill profile (library / DB)

Skill nature
FRAMEWORK
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
5
Sub-category id
1021
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • ETL and ELT Tooling Catalog dimension db id 24

    Library dimension (catalog)

    Roles linked in library: Data Engineer

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
ETL and ELT Tooling
etl-and-elt-tooling
Existing dimension (library) · Role↔dimension saved
Hive Primary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Hive id=2754 · hive

Aliases — catalog

  • Hive (CANONICAL) primary

Context tags (catalog)

Apache Apache Hive Bucketing ETL HQL Hive Metastore Hive SerDe HiveQL MapReduce SQL SQL-on-Hadoop big data bucketing columnar storage data lakes data warehousing integration metadata partitioning schema evolution

Stored enrichment (catalog DB)

Category
Datastore
Sub-category
Local Key Value Store
Vendor
Apache Software Foundation
License
apache_2
Year introduced
2010
Confidence
0.90
Version strategy
NOT_APPLICABLE

Maturity reasoning: Hive appears in Flutter/mobile JDs and package docs, but JD volume is far below SQLite/Realm and it’s mainly used for local key-value storage in Flutter apps.

Skill profile (library / DB)

Skill nature
TOOL
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
3
Sub-category id
2242
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • Local Persistence and Offline Behavior Catalog dimension db id 85

    Library dimension (catalog)

    Roles linked in library: Android Developer, Flutter Developer, Hybrid Mobile Developer, Native Mobile Developer, React Native Developer, iOS Developer

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
Local Persistence and Offline Behavior
local-persistence-and-offline-behavior
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Scala Primary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Scala id=102 · scala

Aliases — catalog

  • Scala (CANONICAL) primary

Context tags (catalog)

Akka Apache Kafka Cats Flink JVM Monads Play Framework SBT ScalaTest Shapeless Spark Spark SQL ZIO case class for-comprehension functional programming implicit pattern matching typeclass

Stored enrichment (catalog DB)

Category
Language
Sub-category
Programming Language
Vendor
EPFL
License
apache_2
Year introduced
2004
Confidence
0.99
Version strategy
NOT_APPLICABLE

Maturity reasoning: Scala still appears in many backend/data engineering JDs, especially with Spark and Akka, and remains supported by major JVM ecosystems; it’s not a sunset technology.

Skill profile (library / DB)

Skill nature
LANGUAGE
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
6
Sub-category id
96
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • Programming Languages for Data Work Catalog dimension db id 21

    Library dimension (catalog)

    Roles linked in library: Data Engineer

  • Programming Languages for ML Systems Catalog dimension db id 39

    Library dimension (catalog)

    Roles linked in library: ML Engineer, MLOps Engineer

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
Programming Languages for Data Work
programming-languages-for-data-work
Existing dimension (library) · Role↔dimension saved
Programming Languages for ML Systems
programming-languages-for-ml-systems
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Hadoop Primary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Hadoop id=1351 · hadoop

Aliases — catalog

  • Hadoop (CANONICAL)

Context tags (catalog)

Big Data Data Lake Distributed Computing ELT ETL Flume HDFS Hive Kafka MapReduce NoSQL Oozie Pig Spark Sqoop YARN

Stored enrichment (catalog DB)

Category
Framework
Sub-category
Data Processing Framework
Vendor
Apache Software Foundation
License
apache_2
Year introduced
2006
Confidence
0.90
Version strategy
NOT_APPLICABLE

Maturity reasoning: Job postings still mention Hadoop for legacy big-data stacks, but JD volume has fallen as Spark and cloud warehouses replaced MapReduce-era clusters.

Skill profile (library / DB)

Skill nature
FRAMEWORK
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
5
Sub-category id
91
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • ETL and ELT Tooling Catalog dimension db id 24

    Library dimension (catalog)

    Roles linked in library: Data Engineer

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
ETL and ELT Tooling
etl-and-elt-tooling
Existing dimension (library) · Role↔dimension saved
EMR Primary New / orchestrated API 3: new canonical path (new) New / unmatched skill (orchestrated in API 2)

Skill enrichment (orchestrator / LLM)

No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).

Derived legacy fields
Category
Cloud Platforms
Sub-category
general
Skill nature
PLATFORM
Volatility
MEDIUM
Typical lifespan
MULTI_YEAR
Version strategy
UNVERSIONED
NoSQL Primary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: NoSQL id=1346 · nosql

Aliases — catalog

  • NoSQL (CANONICAL)

Context tags (catalog)

CAP theorem Cassandra DynamoDB MongoDB Redis column-family data modeling document store eventual consistency graph database horizontal scaling key-value store query language schema-less sharding

Stored enrichment (catalog DB)

Category
Concept
Sub-category
Database Paradigm
Confidence
0.93
Version strategy
NOT_APPLICABLE

Maturity reasoning: NoSQL is broadly listed in job descriptions across backend/data roles, with MongoDB, DynamoDB, and Cassandra appearing as common market signals; it remains a hiring-pipeline staple rather than a niche or sunset tech.

Skill profile (library / DB)

Skill nature
CONCEPT
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
2
Sub-category id
1019
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • NoSQL Databases Catalog dimension db id 19

    Library dimension (catalog)

    Roles linked in library: Backend Developer

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
NoSQL Databases
nosql-databases
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Unix Primary New / orchestrated API 3: new canonical path (new) New / unmatched skill (orchestrated in API 2)

Skill enrichment (orchestrator / LLM)

No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).

Derived legacy fields
Category
Operating Systems
Sub-category
general
Skill nature
TOOL
Volatility
STABLE
Typical lifespan
EVERGREEN
Version strategy
UNVERSIONED
Shell Primary New / orchestrated API 3: new canonical path (new) New / unmatched skill (orchestrated in API 2)

Skill enrichment (orchestrator / LLM)

No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).

Derived legacy fields
Category
Scripting Languages
Sub-category
general
Skill nature
LANGUAGE
Volatility
STABLE
Typical lifespan
EVERGREEN
Version strategy
UNVERSIONED
Oozie Secondary New / orchestrated API 3: new canonical path (new) New / unmatched skill (orchestrated in API 2)

Skill enrichment (orchestrator / LLM)

No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).

Derived legacy fields
Category
Data Engineering Tools
Sub-category
general
Skill nature
TOOL
Volatility
FAST
Typical lifespan
SHORT_LIVED
Version strategy
VERSIONED
Airflow Secondary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Airflow id=265 · airflow

Aliases — catalog

  • Airflow (CANONICAL) primary
  • airflow 2 (VERSION)
  • airflow-2 (VERSION)
  • airflow2 (VERSION)
  • airflow2.x (VERSION)
  • apache airflow 2 (VERSION)

Context tags (catalog)

Apache Celery CeleryExecutor DAG ETL Executor Jinja templating Python SLA Sensors UI XCom backfill connections data pipeline executor hooks logging monitoring operators plugins scheduler task dependencies task instance variables

Stored enrichment (catalog DB)

Category
Tool
Sub-category
Workflow Orchestration Tool
Vendor
Apache Software Foundation
License
apache_2
Year introduced
2014
Confidence
0.95
Version strategy
SEPARATE_ENTITY
Version tag
2.x

Maturity reasoning: Apache Airflow appears in many data engineering job postings and is a common orchestration choice in production stacks; its GitHub activity and ecosystem remain strong, with no vendor sunset or clear replacement dominating JDs.

Skill profile (library / DB)

Skill nature
TOOL
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
13
Sub-category id
130
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • Workflow Orchestration for ML Pipelines Catalog dimension db id 54

    Library dimension (catalog)

    Roles linked in library: ML Engineer, MLOps Engineer

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
Workflow Orchestration for ML Pipelines
workflow-orchestration-for-ml-pipelines
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Apache Storm Secondary New / orchestrated API 3: new canonical path (new) New / unmatched skill (orchestrated in API 2)

Skill enrichment (orchestrator / LLM)

No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).

Derived legacy fields
Category
Data Engineering Tools
Sub-category
general
Skill nature
TOOL
Volatility
MEDIUM
Typical lifespan
MULTI_YEAR
Version strategy
UNVERSIONED
Spark Streaming Secondary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Spark Streaming id=121 · spark-streaming

Aliases — catalog

  • DStreams (VERSION)
  • Spark 2.x (VERSION)
  • Spark 3.x (VERSION)
  • Spark Streaming (VERSION)
  • Spark Structured Streaming (VERSION)
  • Structured Streaming (VERSION)

Context tags (catalog)

DStreams Kafka Kinesis Structured Streaming backpressure checkpointing event time exactly-once micro-batch stateful processing streaming ETL trigger intervals watermarking window functions windowing

Stored enrichment (catalog DB)

Category
Framework
Sub-category
Stream Processing Framework
Vendor
Apache Software Foundation
License
apache_2
Year introduced
2013
Confidence
0.90
Version strategy
SEPARATE_ENTITY
Version tag
Structured Streaming (Spark 2.0+)

Maturity reasoning: JD volume is far lower than Structured Streaming; most Spark streaming roles now specify Structured Streaming or Kafka/Flink, and Spark docs position Spark Streaming as the older API.

Skill profile (library / DB)

Skill nature
FRAMEWORK
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
5
Sub-category id
94
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • Stream Processing Systems Catalog dimension db id 25

    Library dimension (catalog)

    Roles linked in library: Data Engineer

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
Stream Processing Systems
stream-processing-systems
Existing dimension (library) · Role↔dimension saved
NiFi Secondary New / orchestrated API 3: new canonical path (new) New / unmatched skill (orchestrated in API 2)

Skill enrichment (orchestrator / LLM)

No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).

Derived legacy fields
Category
Data Engineering Tools
Sub-category
general
Skill nature
TOOL
Volatility
MEDIUM
Typical lifespan
MULTI_YEAR
Version strategy
UNVERSIONED
Tableau Secondary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Tableau id=150 · tableau

Aliases — catalog

  • Tableau (CANONICAL) primary

Context tags (catalog)

LOD expressions Tableau Cloud Tableau Desktop Tableau Prep Tableau Server actions calculated fields dashboards data blending data visualization extracts filters parameters published data sources workbooks

Stored enrichment (catalog DB)

Category
Platform
Sub-category
Bi Analytics Platform
Vendor
Tableau Software
License
proprietary
Year introduced
2003
Confidence
0.96
Version strategy
NOT_APPLICABLE

Maturity reasoning: Tableau appears frequently in BI/data analyst job descriptions and remains a standard enterprise analytics platform with strong vendor support and broad adoption.

Skill profile (library / DB)

Skill nature
PLATFORM
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
9
Sub-category id
111
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • BI and Visualization Tools Catalog dimension db id 31

    Library dimension (catalog)

    Roles linked in library: Data Engineer

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
BI and Visualization Tools
bi-and-visualization-tools
Existing dimension (library) · Role↔dimension saved
Power BI Secondary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Power BI id=151 · power-bi

Aliases — catalog

  • Power BI (CANONICAL) primary

Context tags (catalog)

Azure Synapse DAX DirectQuery Import mode M language Power Query RLS SQL Server SSAS dashboard data modeling data warehouse gateway reporting star schema

Stored enrichment (catalog DB)

Category
Platform
Sub-category
Bi Analytics Platform
Vendor
Microsoft
License
proprietary
Year introduced
2015
Confidence
0.96
Version strategy
NOT_APPLICABLE

Maturity reasoning: Power BI appears frequently in BI/data analyst job descriptions and is a standard Microsoft analytics platform in enterprise stacks, with strong vendor support and broad adoption.

Skill profile (library / DB)

Skill nature
PLATFORM
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
9
Sub-category id
111
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • BI and Visualization Tools Catalog dimension db id 31

    Library dimension (catalog)

    Roles linked in library: Data Engineer

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
BI and Visualization Tools
bi-and-visualization-tools
Existing dimension (library) · Role↔dimension saved
D3 Secondary New / orchestrated API 3: new canonical path (new) New / unmatched skill (orchestrated in API 2)

Skill enrichment (orchestrator / LLM)

No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).

Derived legacy fields
Category
Data Visualization Tools
Sub-category
general
Skill nature
TOOL
Volatility
MEDIUM
Typical lifespan
MULTI_YEAR
Version strategy
UNVERSIONED
Git Secondary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Git id=1002 · git

Aliases — catalog

  • Git (CANONICAL)

Context tags (catalog)

CI/CD GitHub GitLab branching checkout clone commit fork merging pull request rebase remote repository stash versioning

Stored enrichment (catalog DB)

Category
Tool
Sub-category
Version Control Tool
Vendor
Linus Torvalds
License
gpl_v2
Year introduced
2005
Confidence
0.99
Version strategy
NOT_APPLICABLE

Maturity reasoning: Git is a hiring-pipeline staple: it appears in the vast majority of software engineering job descriptions and is the default VCS on GitHub/GitLab/Bitbucket.

Skill profile (library / DB)

Skill nature
TOOL
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
13
Sub-category id
730
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • React Frontend Development Catalog dimension db id 96

    Library dimension (catalog)

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
React Frontend Development
d_init_01
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
CI/CD Secondary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: CI/CD id=1190 · ci-cd

Aliases — catalog

  • CI/CD (CANONICAL)

Context tags (catalog)

Ansible CircleCI Docker GitLab CI Jenkins Kubernetes Terraform Travis CI automated testing build automation continuous deployment continuous integration deployment pipelines monitoring version control

Stored enrichment (catalog DB)

Category
Methodology
Sub-category
Ci Cd Process
Confidence
0.93
Version strategy
NOT_APPLICABLE

Maturity reasoning: CI/CD appears in a large share of software engineering JDs and is a standard requirement across DevOps, platform, and backend roles; major vendors like GitHub, GitLab, and AWS all center product roadmaps on CI/CD pipelines.

Skill profile (library / DB)

Skill nature
METHODOLOGY
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
8
Sub-category id
900
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • CI/CD Pipeline Platforms Catalog dimension db id 150

    Library dimension (catalog)

    Roles linked in library: DevOps Engineer

  • CI/CD for Machine Learning Catalog dimension db id 56

    Library dimension (catalog)

    Roles linked in library: ML Engineer

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
CI/CD Pipeline Platforms
ci-cd-pipeline-platforms
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
CI/CD for Machine Learning
ci-cd-for-machine-learning
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Master Data Management Secondary New / orchestrated API 3: new canonical path (new) New / unmatched skill (orchestrated in API 2)

Skill enrichment (orchestrator / LLM)

No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).

Derived legacy fields
Category
Data Management
Sub-category
general
Skill nature
CONCEPT
Volatility
MEDIUM
Typical lifespan
MULTI_YEAR
Version strategy
UNVERSIONED
Machine Learning Secondary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Machine Learning id=1356 · machine-learning

Aliases — catalog

  • Machine Learning (CANONICAL)

Context tags (catalog)

Keras PyTorch TensorFlow cross-validation data preprocessing ensemble methods feature engineering hyperparameter tuning model evaluation natural language processing neural networks reinforcement learning scikit-learn supervised learning unsupervised learning

Stored enrichment (catalog DB)

Category
Concept
Sub-category
Machine Learning
Confidence
0.98
Version strategy
NOT_APPLICABLE

Maturity reasoning: Machine Learning appears in large volumes of job descriptions across data, product, and platform roles, and major cloud vendors (AWS, Google Cloud, Azure) offer dedicated ML services and certifications, indicating broad adoption.

Skill profile (library / DB)

Skill nature
CONCEPT
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
2
Sub-category id
1024
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • AI Governance and Model Security Catalog dimension db id 50

    Library dimension (catalog)

    Roles linked in library: AI Engineer, ML Engineer, MLOps Engineer

  • React Frontend Development Catalog dimension db id 96

    Library dimension (catalog)

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
AI Governance and Model Security
ai-governance-and-model-security
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
React Frontend Development
d_init_01
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
SOA Secondary New / orchestrated API 3: new canonical path (new) New / unmatched skill (orchestrated in API 2)

Skill enrichment (orchestrator / LLM)

No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).

Derived legacy fields
Category
Architecture Concepts
Sub-category
general
Skill nature
CONCEPT
Volatility
MEDIUM
Typical lifespan
MULTI_YEAR
Version strategy
UNVERSIONED
API Development Secondary New / orchestrated API 3: new canonical path (new) New / unmatched skill (orchestrated in API 2)

Skill enrichment (orchestrator / LLM)

No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).

Derived legacy fields
Category
API Tools
Sub-category
general
Skill nature
PRACTICE
Volatility
MEDIUM
Typical lifespan
MULTI_YEAR
Version strategy
UNVERSIONED
Agile Secondary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Agile id=520 · agile

Aliases — catalog

  • Agile (CANONICAL) primary

Context tags (catalog)

Kanban SAFe Scrum backlog backlog grooming burndown burndown chart continuous delivery continuous improvement cross-functional daily standup epics incremental development iteration iteration planning lean product backlog product owner retrospective sprint sprint planning stand-up story points user stories velocity

Stored enrichment (catalog DB)

Category
Methodology
Sub-category
Agile
Confidence
0.99
Version strategy
NOT_APPLICABLE

Maturity reasoning: Agile appears in a large share of software job descriptions and is a standard hiring-pipeline requirement; Scrum/Kanban are commonly listed alongside it, showing broad market adoption.

Skill profile (library / DB)

Skill nature
METHODOLOGY
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
8
Sub-category id
367
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • React Frontend Development Catalog dimension db id 96

    Library dimension (catalog)

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
React Frontend Development
d_init_01
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Scrum Secondary Library skill API 3: existing canonical (in_db) Existing skill (matched library)
Canonical: Scrum id=1215 · scrum

Aliases — catalog

  • Scrum (CANONICAL) primary

Context tags (catalog)

Agile Kanban backlog burndown chart cross-functional team daily standup definition of done increment product owner release planning retrospective scrum framework scrum master sprint story points user stories velocity

Stored enrichment (catalog DB)

Category
Methodology
Sub-category
Scrum
Confidence
0.99
Version strategy
NOT_APPLICABLE

Maturity reasoning: Scrum appears in a large share of agile project-management and product-owner job descriptions, and Scrum Alliance/PSM certifications are widely requested in hiring pipelines.

Skill profile (library / DB)

Skill nature
METHODOLOGY
Volatility
STABLE
Typical lifespan
EVERGREEN
Category id
8
Sub-category id
921
Extractable
True
Also category
False

Dimensions (API 2 worklist)

  • React Frontend Development Catalog dimension db id 96

    Library dimension (catalog)

API 3 link attempts (this skill)

Dimension Skill↔dim Role↔dim Outcome
React Frontend Development
d_init_01
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
MLlib Secondary New / orchestrated API 3: new canonical path (new) New / unmatched skill (orchestrated in API 2)

Skill enrichment (orchestrator / LLM)

No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).

Derived legacy fields
Category
Machine Learning Frameworks
Sub-category
general
Skill nature
TOOL
Volatility
MEDIUM
Typical lifespan
MULTI_YEAR
Version strategy
UNVERSIONED
GraphX Secondary New / orchestrated API 3: new canonical path (new) New / unmatched skill (orchestrated in API 2)

Skill enrichment (orchestrator / LLM)

No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).

Derived legacy fields
Category
Machine Learning Frameworks
Sub-category
general
Skill nature
TOOL
Volatility
MEDIUM
Typical lifespan
MULTI_YEAR
Version strategy
UNVERSIONED
Pandas Secondary New / orchestrated API 3: new canonical path (new) New / unmatched skill (orchestrated in API 2)

Skill enrichment (orchestrator / LLM)

No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).

Derived legacy fields
Category
Data Analysis Libraries
Sub-category
general
Skill nature
TOOL
Volatility
STABLE
Typical lifespan
EVERGREEN
Version strategy
UNVERSIONED

All API 3 persistence rows

Same grid as the skill-extractor “Persistence items” table: one row per (skill × dimension) work item.

Skill Tag Dimension Skill↔dim Role↔dim Outcome Notes
SQL in_db
Pega Programming Languages & DSLs
pega-programming-languages-dsls
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
SQL in_db
Programming Languages for Data Work
programming-languages-for-data-work
Existing dimension (library) · Role↔dimension saved
Python in_db
Cloud Security Scripting & DSL Languages
cloud-security-scripting-dsl-languages
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Python in_db
Programming Languages
programming-languages
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Python in_db
Programming Languages and Scripting
programming-languages-and-scripting
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Python in_db
Programming Languages for Data Work
programming-languages-for-data-work
Existing dimension (library) · Role↔dimension saved
Python in_db
Programming Languages for ML Systems
programming-languages-for-ml-systems
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Python in_db
Programming Languages for XR
programming-languages-for-xr
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Python in_db
Python Programming
python-programming
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Apache Spark in_db
ETL and ELT Tooling
etl-and-elt-tooling
Existing dimension (library) · Role↔dimension saved
Hive in_db
Local Persistence and Offline Behavior
local-persistence-and-offline-behavior
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Scala in_db
Programming Languages for Data Work
programming-languages-for-data-work
Existing dimension (library) · Role↔dimension saved
Scala in_db
Programming Languages for ML Systems
programming-languages-for-ml-systems
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Hadoop in_db
ETL and ELT Tooling
etl-and-elt-tooling
Existing dimension (library) · Role↔dimension saved
NoSQL in_db
NoSQL Databases
nosql-databases
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Airflow in_db
Workflow Orchestration for ML Pipelines
workflow-orchestration-for-ml-pipelines
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Spark Streaming in_db
Stream Processing Systems
stream-processing-systems
Existing dimension (library) · Role↔dimension saved
Tableau in_db
BI and Visualization Tools
bi-and-visualization-tools
Existing dimension (library) · Role↔dimension saved
Power BI in_db
BI and Visualization Tools
bi-and-visualization-tools
Existing dimension (library) · Role↔dimension saved
Git in_db
React Frontend Development
d_init_01
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
CI/CD in_db
CI/CD Pipeline Platforms
ci-cd-pipeline-platforms
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
CI/CD in_db
CI/CD for Machine Learning
ci-cd-for-machine-learning
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Machine Learning in_db
AI Governance and Model Security
ai-governance-and-model-security
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Machine Learning in_db
React Frontend Development
d_init_01
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Agile in_db
React Frontend Development
d_init_01
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)
Scrum in_db
React Frontend Development
d_init_01
Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role)

Library artifacts (this run)

Kind Detail DB id
canonical_skill_proposed Data Engineering | type=Data Engineering Tools subtype=general nature=PRACTICE lifespan=MULTI_YEAR
canonical_skill_proposed Data Architecture | type=Data Engineering Tools subtype=general nature=CONCEPT lifespan=MULTI_YEAR
canonical_skill_proposed EMR | type=Cloud Platforms subtype=general nature=PLATFORM lifespan=MULTI_YEAR
canonical_skill_proposed Unix | type=Operating Systems subtype=general nature=TOOL lifespan=EVERGREEN
canonical_skill_proposed Shell | type=Scripting Languages subtype=general nature=LANGUAGE lifespan=EVERGREEN
canonical_skill_proposed Oozie | type=Data Engineering Tools subtype=general nature=TOOL lifespan=SHORT_LIVED
canonical_skill_proposed Apache Storm | type=Data Engineering Tools subtype=general nature=TOOL lifespan=MULTI_YEAR
canonical_skill_proposed NiFi | type=Data Engineering Tools subtype=general nature=TOOL lifespan=MULTI_YEAR
canonical_skill_proposed D3 | type=Data Visualization Tools subtype=general nature=TOOL lifespan=MULTI_YEAR
canonical_skill_proposed Master Data Management | type=Data Management subtype=general nature=CONCEPT lifespan=MULTI_YEAR
canonical_skill_proposed SOA | type=Architecture Concepts subtype=general nature=CONCEPT lifespan=MULTI_YEAR
canonical_skill_proposed API Development | type=API Tools subtype=general nature=PRACTICE lifespan=MULTI_YEAR
canonical_skill_proposed MLlib | type=Machine Learning Frameworks subtype=general nature=TOOL lifespan=MULTI_YEAR
canonical_skill_proposed GraphX | type=Machine Learning Frameworks subtype=general nature=TOOL lifespan=MULTI_YEAR
canonical_skill_proposed Pandas | type=Data Analysis Libraries subtype=general nature=TOOL lifespan=EVERGREEN
nano JD Parser — gpt-4.1-nano click to toggle
RoleLead Data Engineer
Experience5+ years of experience in Data Engineering
DomainOther
JD type pass
Show raw JSON
{
  "JD_type": "pass",
  "about_company": null,
  "certifications": [],
  "company_name": null,
  "ctc": null,
  "domain": {
    "primary": {
      "aliases": [],
      "domain": "Other"
    },
    "secondary": null
  },
  "education": [],
  "experience": {
    "max": null,
    "min": 5,
    "raw": "5+ years of experience in Data Engineering"
  },
  "job_locations": [],
  "role": "Lead Data Engineer",
  "role_aliases": [
    "Data Engineer",
    "Senior Data Engineer",
    "Data Engineering Lead"
  ],
  "role_archetype": "Data",
  "roles_and_responsibilities": [
    {
      "bullet_count": 0,
      "heading": "Responsibilities",
      "heading_was_present": true,
      "source_marker": {
        "first_5_words": "As Lead Data Engineer you",
        "last_5_words": "technical to the highly creative."
      },
      "text": "As Lead Data Engineer you will design and implement the next version of our integrated data platform.You will own and drive strategic planning, business analysis and technical implementation of data engineering, tools, and data architecture definition.In addition to managing our portfolio of datasets globally, you will play key role in leading our data engineering community to leverage these datasets effectively.You will also produce documentation and lead training sessions for data engineering solutions.Collaborate with and influence leadership so that your teams work directly impacts company strategy and direction.Develop new techniques and data that will enable answering previously unanswerable and new questions.Develop deep partnerships with engineering and product teams to deliver on major cross-functional measurements, testing.Communicate effectively to all levels of the organization, including executives.Provide success in partnering teams with dramatically varying backgrounds, from the highly technical to the highly creative.",
      "word_count": 194
    },
    {
      "bullet_count": 0,
      "heading": "Requirements",
      "heading_was_present": true,
      "source_marker": {
        "first_5_words": "Experience designing a data engineering",
        "last_5_words": "business cases."
      },
      "text": "Experience designing a data engineering roadmap and executing the vision behind it.Experience instilling a culture of ownership, collaboration, and results.Track record of recruiting talent in data engineering, analytics, and data science.Good business acumen to orient data analysis to business needs of clients.Ability to translate business concepts into a technical architectureAbility to collaborate with cross-functional teams to define priorities and develop roadmapsProject management skillsDemonstrated intellectual and analytical rigor, strong attention to detail Team oriented, energetic, collaborative, diplomatic, and flexible styleYou will have the opportunity to be very autonomous, self-driven, able to navigate and lead through ambiguityYou excel at articulating complex topics to the layman (understands audience and speaks to be understood)Experience with crafting and operating scalable, resilient, highly available servicesYou have a deep understanding and appreciation of both machine-readable standards and master data management (MDM)Work experience in an analytical field such as computer science.5+ years of experience in Data EngineeringWorking knowledge of Hadoop ecosystem and associated technologies, (For e. g., Apache Spark, MLlib, GraphX, iPython, sci-kit, Pandas etc. )Experience in creating large scale data engineering pipelines, data-based decision-making, and quantitative analysis.Exposure with Visualization Tools like Tableau, Power BI, D3 and exposure to code version control systems (git).Experience - CICD/ Data Architectures/ Pipelines/ Quality /Code Management.Advanced experience in writing and optimizing efficient SQL queries with Python, Spark, Hive, Scala handling Large Data Sets in Big-Data Environments.Experience with SQL for extracting, aggregating, and processing big data Pipelines using Hadoop, EMR \u0026 NoSQL Databases.Experience creating/supporting production software/systems and a proven track record of identifying and resolving performance bottlenecks for production systems.Experience with Unix/Shell and exposure to Scheduling tools like Oozie and Airflow.Exposure to stream-processing systems like Apache Storm, Spark-Streaming.Experience with Big Data platforms such as Apache Hadoop and Apache Spark, NiFiUnderstanding of Machine Learning technology concepts and applicationExperience implementing SOA frameworkExperience with enterprise library development and management Experience with API development and managementExperience working in agile scrum environmentAbility to translate data and technical concepts into requirements documents, business cases.",
      "word_count": 469
    }
  ],
  "urls": []
}
API 1 — extract-from-jd click to toggle
{
  "final_skills": [
    {
      "is_primary": true,
      "skill_name": "Data Engineering"
    },
    {
      "is_primary": true,
      "skill_name": "Data Architecture"
    },
    {
      "is_primary": true,
      "skill_name": "SQL"
    },
    {
      "is_primary": true,
      "skill_name": "Python"
    },
    {
      "is_primary": true,
      "skill_name": "Apache Spark"
    },
    {
      "is_primary": true,
      "skill_name": "Hive"
    },
    {
      "is_primary": true,
      "skill_name": "Scala"
    },
    {
      "is_primary": true,
      "skill_name": "Hadoop"
    },
    {
      "is_primary": true,
      "skill_name": "EMR"
    },
    {
      "is_primary": true,
      "skill_name": "NoSQL"
    },
    {
      "is_primary": true,
      "skill_name": "Unix"
    },
    {
      "is_primary": true,
      "skill_name": "Shell"
    },
    {
      "is_primary": false,
      "skill_name": "Oozie"
    },
    {
      "is_primary": false,
      "skill_name": "Airflow"
    },
    {
      "is_primary": false,
      "skill_name": "Apache Storm"
    },
    {
      "is_primary": false,
      "skill_name": "Spark Streaming"
    },
    {
      "is_primary": false,
      "skill_name": "NiFi"
    },
    {
      "is_primary": false,
      "skill_name": "Tableau"
    },
    {
      "is_primary": false,
      "skill_name": "Power BI"
    },
    {
      "is_primary": false,
      "skill_name": "D3"
    },
    {
      "is_primary": false,
      "skill_name": "Git"
    },
    {
      "is_primary": false,
      "skill_name": "CI/CD"
    },
    {
      "is_primary": false,
      "skill_name": "Master Data Management"
    },
    {
      "is_primary": false,
      "skill_name": "Machine Learning"
    },
    {
      "is_primary": false,
      "skill_name": "SOA"
    },
    {
      "is_primary": false,
      "skill_name": "API Development"
    },
    {
      "is_primary": false,
      "skill_name": "Agile"
    },
    {
      "is_primary": false,
      "skill_name": "Scrum"
    },
    {
      "is_primary": false,
      "skill_name": "MLlib"
    },
    {
      "is_primary": false,
      "skill_name": "GraphX"
    },
    {
      "is_primary": false,
      "skill_name": "Pandas"
    }
  ],
  "jd_role": {
    "display_name": "Lead Data Engineer",
    "rationale": null,
    "role_aliases": [
      "Data Engineer",
      "Senior Data Engineer",
      "Data Engineering Lead"
    ],
    "role_archetype": "Data",
    "slug": ""
  },
  "nano_parsed": {
    "JD_type": "pass",
    "about_company": null,
    "certifications": [],
    "company_name": null,
    "ctc": null,
    "domain": {
      "primary": {
        "aliases": [],
        "domain": "Other"
      },
      "secondary": null
    },
    "education": [],
    "experience": {
      "max": null,
      "min": 5,
      "raw": "5+ years of experience in Data Engineering"
    },
    "job_locations": [],
    "role": "Lead Data Engineer",
    "role_aliases": [
      "Data Engineer",
      "Senior Data Engineer",
      "Data Engineering Lead"
    ],
    "role_archetype": "Data",
    "roles_and_responsibilities": [
      {
        "bullet_count": 0,
        "heading": "Responsibilities",
        "heading_was_present": true,
        "source_marker": {
          "first_5_words": "As Lead Data Engineer you",
          "last_5_words": "technical to the highly creative."
        },
        "text": "As Lead Data Engineer you will design and implement the next version of our integrated data platform.You will own and drive strategic planning, business analysis and technical implementation of data engineering, tools, and data architecture definition.In addition to managing our portfolio of datasets globally, you will play key role in leading our data engineering community to leverage these datasets effectively.You will also produce documentation and lead training sessions for data engineering solutions.Collaborate with and influence leadership so that your teams work directly impacts company strategy and direction.Develop new techniques and data that will enable answering previously unanswerable and new questions.Develop deep partnerships with engineering and product teams to deliver on major cross-functional measurements, testing.Communicate effectively to all levels of the organization, including executives.Provide success in partnering teams with dramatically varying backgrounds, from the highly technical to the highly creative.",
        "word_count": 194
      },
      {
        "bullet_count": 0,
        "heading": "Requirements",
        "heading_was_present": true,
        "source_marker": {
          "first_5_words": "Experience designing a data engineering",
          "last_5_words": "business cases."
        },
        "text": "Experience designing a data engineering roadmap and executing the vision behind it.Experience instilling a culture of ownership, collaboration, and results.Track record of recruiting talent in data engineering, analytics, and data science.Good business acumen to orient data analysis to business needs of clients.Ability to translate business concepts into a technical architectureAbility to collaborate with cross-functional teams to define priorities and develop roadmapsProject management skillsDemonstrated intellectual and analytical rigor, strong attention to detail Team oriented, energetic, collaborative, diplomatic, and flexible styleYou will have the opportunity to be very autonomous, self-driven, able to navigate and lead through ambiguityYou excel at articulating complex topics to the layman (understands audience and speaks to be understood)Experience with crafting and operating scalable, resilient, highly available servicesYou have a deep understanding and appreciation of both machine-readable standards and master data management (MDM)Work experience in an analytical field such as computer science.5+ years of experience in Data EngineeringWorking knowledge of Hadoop ecosystem and associated technologies, (For e. g., Apache Spark, MLlib, GraphX, iPython, sci-kit, Pandas etc. )Experience in creating large scale data engineering pipelines, data-based decision-making, and quantitative analysis.Exposure with Visualization Tools like Tableau, Power BI, D3 and exposure to code version control systems (git).Experience - CICD/ Data Architectures/ Pipelines/ Quality /Code Management.Advanced experience in writing and optimizing efficient SQL queries with Python, Spark, Hive, Scala handling Large Data Sets in Big-Data Environments.Experience with SQL for extracting, aggregating, and processing big data Pipelines using Hadoop, EMR \u0026 NoSQL Databases.Experience creating/supporting production software/systems and a proven track record of identifying and resolving performance bottlenecks for production systems.Experience with Unix/Shell and exposure to Scheduling tools like Oozie and Airflow.Exposure to stream-processing systems like Apache Storm, Spark-Streaming.Experience with Big Data platforms such as Apache Hadoop and Apache Spark, NiFiUnderstanding of Machine Learning technology concepts and applicationExperience implementing SOA frameworkExperience with enterprise library development and management Experience with API development and managementExperience working in agile scrum environmentAbility to translate data and technical concepts into requirements documents, business cases.",
        "word_count": 469
      }
    ],
    "urls": []
  },
  "rejected": false,
  "rejection_reason": null,
  "run_id": "459d8d5b-8e49-4c9b-a801-9389a9c4c4ae",
  "stage3_signals": {
    "alias_found": true,
    "alias_match_roles": [
      {
        "display_name": "Data Engineer",
        "kra_matches": null,
        "matched_count": null,
        "matched_skills": null,
        "role_id": 2,
        "score": 1.0,
        "slug": "data-engineer",
        "total_count": null
      }
    ],
    "kra_match_roles": [
      {
        "display_name": "Data Engineer",
        "kra_matches": [
          {
            "kra_text": "Works with data analysts, data scientists, and business stakeholders to define data models, ingestion schedules, and data delivery requirements.",
            "sentence": "As Lead Data Engineer you will design and implement the next version of our integrated data platform.You will own and drive strategic planning, business analysis and technical implementation of data engineering, tools, and data architecture definition.In addition to managing our portfolio of datasets globally, you will play key role in leading our data engineering community to leverage these datasets effectively.You will also produce documentation and lead training sessions for data engineering solutions.Collaborate with and influence leadership so that your teams work directly impacts company strategy and direction.Develop new techniques and data that will enable answering previously unanswerable and new questions.Develop deep partnerships with engineering and product teams to deliver on major cross-functional measurements, testing.Communicate effectively to all levels of the organization, including executives.Provide success in partnering teams with dramatically varying backgrounds, from the highly technical to the highly creative.",
            "similarity": 0.5644
          },
          {
            "kra_text": "Works with data analysts, data scientists, and business stakeholders to define data models, ingestion schedules, and data delivery requirements.",
            "sentence": "Experience designing a data engineering roadmap and executing the vision behind it.Experience instilling a culture of ownership, collaboration, and results.Track record of recruiting talent in data engineering, analytics, and data science.Good business acumen to orient data analysis to business needs of clients.Ability to translate business concepts into a technical architectureAbility to collaborate with cross-functional teams to define priorities and develop roadmapsProject management skillsDemonstrated intellectual and analytical rigor, strong attention to detail Team oriented, energetic, collaborative, diplomatic, and flexible styleYou will have the opportunity to be very autonomous, self-driven, able to navigate and lead through ambiguityYou excel at articulating complex topics to the layman (understands audience and speaks to be understood)Experience with crafting and operating scalable, resilient, highly available servicesYou have a deep understanding and appreciation of both machine-readable standards and master data management (MDM)Work experience in an analytical field such as computer science.5+ years of experience in Data EngineeringWorking knowledge of Hadoop ecosystem and associated technologies, (For e. g., Apache Spark, MLlib, GraphX, iPython, sci-kit, Pandas etc. )Experience in creating large scale data engineering pipelines, data-based decision-making, and quantitative analysis.Exposure with Visualization Tools like Tableau, Power BI, D3 and exposure to code version control systems (git).Experience - CICD/ Data Architectures/ Pipelines/ Quality /Code Management.Advanced experience in writing and optimizing efficient SQL queries with Python, Spark, Hive, Scala handling Large Data Sets in Big-Data Environments.Experience with SQL for extracting, aggregating, and processing big data Pipelines using Hadoop, EMR \u0026 NoSQL Databases.Experience creating/supporting production software/systems and a proven track record of identifying and resolving performance bottlenecks for production systems.Experience with Unix/Shell and exposure to Scheduling tools like Oozie and Airflow.Exposure to stream-processing systems like Apache Storm, Spark-Streaming.Experience with Big Data platforms such as Apache Hadoop and Apache Spark, NiFiUnderstanding of Machine Learning technology concepts and applicationExperience implementing SOA frameworkExperience with enterprise library development and management Experience with API development and managementExperience working in agile scrum environmentAbility to translate data and technical concepts into requirements documents, business cases.",
            "similarity": 0.5619
          }
        ],
        "matched_count": null,
        "matched_skills": null,
        "role_id": 2,
        "score": 0.5632,
        "slug": "data-engineer",
        "total_count": null
      },
      {
        "display_name": "ML Engineer",
        "kra_matches": [
          {
            "kra_text": "Prepares, cleans, and transforms training datasets, manages feature stores, and builds feature engineering pipelines for model training.",
            "sentence": "As Lead Data Engineer you will design and implement the next version of our integrated data platform.You will own and drive strategic planning, business analysis and technical implementation of data engineering, tools, and data architecture definition.In addition to managing our portfolio of datasets globally, you will play key role in leading our data engineering community to leverage these datasets effectively.You will also produce documentation and lead training sessions for data engineering solutions.Collaborate with and influence leadership so that your teams work directly impacts company strategy and direction.Develop new techniques and data that will enable answering previously unanswerable and new questions.Develop deep partnerships with engineering and product teams to deliver on major cross-functional measurements, testing.Communicate effectively to all levels of the organization, including executives.Provide success in partnering teams with dramatically varying backgrounds, from the highly technical to the highly creative.",
            "similarity": 0.4731
          },
          {
            "kra_text": "Prepares, cleans, and transforms training datasets, manages feature stores, and builds feature engineering pipelines for model training.",
            "sentence": "Experience designing a data engineering roadmap and executing the vision behind it.Experience instilling a culture of ownership, collaboration, and results.Track record of recruiting talent in data engineering, analytics, and data science.Good business acumen to orient data analysis to business needs of clients.Ability to translate business concepts into a technical architectureAbility to collaborate with cross-functional teams to define priorities and develop roadmapsProject management skillsDemonstrated intellectual and analytical rigor, strong attention to detail Team oriented, energetic, collaborative, diplomatic, and flexible styleYou will have the opportunity to be very autonomous, self-driven, able to navigate and lead through ambiguityYou excel at articulating complex topics to the layman (understands audience and speaks to be understood)Experience with crafting and operating scalable, resilient, highly available servicesYou have a deep understanding and appreciation of both machine-readable standards and master data management (MDM)Work experience in an analytical field such as computer science.5+ years of experience in Data EngineeringWorking knowledge of Hadoop ecosystem and associated technologies, (For e. g., Apache Spark, MLlib, GraphX, iPython, sci-kit, Pandas etc. )Experience in creating large scale data engineering pipelines, data-based decision-making, and quantitative analysis.Exposure with Visualization Tools like Tableau, Power BI, D3 and exposure to code version control systems (git).Experience - CICD/ Data Architectures/ Pipelines/ Quality /Code Management.Advanced experience in writing and optimizing efficient SQL queries with Python, Spark, Hive, Scala handling Large Data Sets in Big-Data Environments.Experience with SQL for extracting, aggregating, and processing big data Pipelines using Hadoop, EMR \u0026 NoSQL Databases.Experience creating/supporting production software/systems and a proven track record of identifying and resolving performance bottlenecks for production systems.Experience with Unix/Shell and exposure to Scheduling tools like Oozie and Airflow.Exposure to stream-processing systems like Apache Storm, Spark-Streaming.Experience with Big Data platforms such as Apache Hadoop and Apache Spark, NiFiUnderstanding of Machine Learning technology concepts and applicationExperience implementing SOA frameworkExperience with enterprise library development and management Experience with API development and managementExperience working in agile scrum environmentAbility to translate data and technical concepts into requirements documents, business cases.",
            "similarity": 0.4551
          }
        ],
        "matched_count": null,
        "matched_skills": null,
        "role_id": 3,
        "score": 0.4641,
        "slug": "ml-engineer",
        "total_count": null
      },
      {
        "display_name": "Cloud Architect",
        "kra_matches": [
          {
            "kra_text": "Conducts architecture reviews, approves technical design documents, and guides engineering teams through cloud migration and modernization projects.",
            "sentence": "As Lead Data Engineer you will design and implement the next version of our integrated data platform.You will own and drive strategic planning, business analysis and technical implementation of data engineering, tools, and data architecture definition.In addition to managing our portfolio of datasets globally, you will play key role in leading our data engineering community to leverage these datasets effectively.You will also produce documentation and lead training sessions for data engineering solutions.Collaborate with and influence leadership so that your teams work directly impacts company strategy and direction.Develop new techniques and data that will enable answering previously unanswerable and new questions.Develop deep partnerships with engineering and product teams to deliver on major cross-functional measurements, testing.Communicate effectively to all levels of the organization, including executives.Provide success in partnering teams with dramatically varying backgrounds, from the highly technical to the highly creative.",
            "similarity": 0.4417
          },
          {
            "kra_text": "Conducts architecture reviews, approves technical design documents, and guides engineering teams through cloud migration and modernization projects.",
            "sentence": "Experience designing a data engineering roadmap and executing the vision behind it.Experience instilling a culture of ownership, collaboration, and results.Track record of recruiting talent in data engineering, analytics, and data science.Good business acumen to orient data analysis to business needs of clients.Ability to translate business concepts into a technical architectureAbility to collaborate with cross-functional teams to define priorities and develop roadmapsProject management skillsDemonstrated intellectual and analytical rigor, strong attention to detail Team oriented, energetic, collaborative, diplomatic, and flexible styleYou will have the opportunity to be very autonomous, self-driven, able to navigate and lead through ambiguityYou excel at articulating complex topics to the layman (understands audience and speaks to be understood)Experience with crafting and operating scalable, resilient, highly available servicesYou have a deep understanding and appreciation of both machine-readable standards and master data management (MDM)Work experience in an analytical field such as computer science.5+ years of experience in Data EngineeringWorking knowledge of Hadoop ecosystem and associated technologies, (For e. g., Apache Spark, MLlib, GraphX, iPython, sci-kit, Pandas etc. )Experience in creating large scale data engineering pipelines, data-based decision-making, and quantitative analysis.Exposure with Visualization Tools like Tableau, Power BI, D3 and exposure to code version control systems (git).Experience - CICD/ Data Architectures/ Pipelines/ Quality /Code Management.Advanced experience in writing and optimizing efficient SQL queries with Python, Spark, Hive, Scala handling Large Data Sets in Big-Data Environments.Experience with SQL for extracting, aggregating, and processing big data Pipelines using Hadoop, EMR \u0026 NoSQL Databases.Experience creating/supporting production software/systems and a proven track record of identifying and resolving performance bottlenecks for production systems.Experience with Unix/Shell and exposure to Scheduling tools like Oozie and Airflow.Exposure to stream-processing systems like Apache Storm, Spark-Streaming.Experience with Big Data platforms such as Apache Hadoop and Apache Spark, NiFiUnderstanding of Machine Learning technology concepts and applicationExperience implementing SOA frameworkExperience with enterprise library development and management Experience with API development and managementExperience working in agile scrum environmentAbility to translate data and technical concepts into requirements documents, business cases.",
            "similarity": 0.4413
          }
        ],
        "matched_count": null,
        "matched_skills": null,
        "role_id": 9,
        "score": 0.4415,
        "slug": "cloud-architect",
        "total_count": null
      },
      {
        "display_name": "Fullstack Developer",
        "kra_matches": [
          {
            "kra_text": "Designs and queries relational databases like PostgreSQL and document stores like MongoDB, writing migrations, indexes, and optimized queries.",
            "sentence": "As Lead Data Engineer you will design and implement the next version of our integrated data platform.You will own and drive strategic planning, business analysis and technical implementation of data engineering, tools, and data architecture definition.In addition to managing our portfolio of datasets globally, you will play key role in leading our data engineering community to leverage these datasets effectively.You will also produce documentation and lead training sessions for data engineering solutions.Collaborate with and influence leadership so that your teams work directly impacts company strategy and direction.Develop new techniques and data that will enable answering previously unanswerable and new questions.Develop deep partnerships with engineering and product teams to deliver on major cross-functional measurements, testing.Communicate effectively to all levels of the organization, including executives.Provide success in partnering teams with dramatically varying backgrounds, from the highly technical to the highly creative.",
            "similarity": 0.4212
          },
          {
            "kra_text": "Designs and queries relational databases like PostgreSQL and document stores like MongoDB, writing migrations, indexes, and optimized queries.",
            "sentence": "Experience designing a data engineering roadmap and executing the vision behind it.Experience instilling a culture of ownership, collaboration, and results.Track record of recruiting talent in data engineering, analytics, and data science.Good business acumen to orient data analysis to business needs of clients.Ability to translate business concepts into a technical architectureAbility to collaborate with cross-functional teams to define priorities and develop roadmapsProject management skillsDemonstrated intellectual and analytical rigor, strong attention to detail Team oriented, energetic, collaborative, diplomatic, and flexible styleYou will have the opportunity to be very autonomous, self-driven, able to navigate and lead through ambiguityYou excel at articulating complex topics to the layman (understands audience and speaks to be understood)Experience with crafting and operating scalable, resilient, highly available servicesYou have a deep understanding and appreciation of both machine-readable standards and master data management (MDM)Work experience in an analytical field such as computer science.5+ years of experience in Data EngineeringWorking knowledge of Hadoop ecosystem and associated technologies, (For e. g., Apache Spark, MLlib, GraphX, iPython, sci-kit, Pandas etc. )Experience in creating large scale data engineering pipelines, data-based decision-making, and quantitative analysis.Exposure with Visualization Tools like Tableau, Power BI, D3 and exposure to code version control systems (git).Experience - CICD/ Data Architectures/ Pipelines/ Quality /Code Management.Advanced experience in writing and optimizing efficient SQL queries with Python, Spark, Hive, Scala handling Large Data Sets in Big-Data Environments.Experience with SQL for extracting, aggregating, and processing big data Pipelines using Hadoop, EMR \u0026 NoSQL Databases.Experience creating/supporting production software/systems and a proven track record of identifying and resolving performance bottlenecks for production systems.Experience with Unix/Shell and exposure to Scheduling tools like Oozie and Airflow.Exposure to stream-processing systems like Apache Storm, Spark-Streaming.Experience with Big Data platforms such as Apache Hadoop and Apache Spark, NiFiUnderstanding of Machine Learning technology concepts and applicationExperience implementing SOA frameworkExperience with enterprise library development and management Experience with API development and managementExperience working in agile scrum environmentAbility to translate data and technical concepts into requirements documents, business cases.",
            "similarity": 0.4063
          }
        ],
        "matched_count": null,
        "matched_skills": null,
        "role_id": 15,
        "score": 0.4137,
        "slug": "full-stack-engineer",
        "total_count": null
      },
      {
        "display_name": "AI Engineer",
        "kra_matches": [
          {
            "kra_text": "Designs and implements prompt engineering workflows, few-shot examples, chain-of-thought patterns, and structured output parsing for AI feature pipelines.",
            "sentence": "Experience designing a data engineering roadmap and executing the vision behind it.Experience instilling a culture of ownership, collaboration, and results.Track record of recruiting talent in data engineering, analytics, and data science.Good business acumen to orient data analysis to business needs of clients.Ability to translate business concepts into a technical architectureAbility to collaborate with cross-functional teams to define priorities and develop roadmapsProject management skillsDemonstrated intellectual and analytical rigor, strong attention to detail Team oriented, energetic, collaborative, diplomatic, and flexible styleYou will have the opportunity to be very autonomous, self-driven, able to navigate and lead through ambiguityYou excel at articulating complex topics to the layman (understands audience and speaks to be understood)Experience with crafting and operating scalable, resilient, highly available servicesYou have a deep understanding and appreciation of both machine-readable standards and master data management (MDM)Work experience in an analytical field such as computer science.5+ years of experience in Data EngineeringWorking knowledge of Hadoop ecosystem and associated technologies, (For e. g., Apache Spark, MLlib, GraphX, iPython, sci-kit, Pandas etc. )Experience in creating large scale data engineering pipelines, data-based decision-making, and quantitative analysis.Exposure with Visualization Tools like Tableau, Power BI, D3 and exposure to code version control systems (git).Experience - CICD/ Data Architectures/ Pipelines/ Quality /Code Management.Advanced experience in writing and optimizing efficient SQL queries with Python, Spark, Hive, Scala handling Large Data Sets in Big-Data Environments.Experience with SQL for extracting, aggregating, and processing big data Pipelines using Hadoop, EMR \u0026 NoSQL Databases.Experience creating/supporting production software/systems and a proven track record of identifying and resolving performance bottlenecks for production systems.Experience with Unix/Shell and exposure to Scheduling tools like Oozie and Airflow.Exposure to stream-processing systems like Apache Storm, Spark-Streaming.Experience with Big Data platforms such as Apache Hadoop and Apache Spark, NiFiUnderstanding of Machine Learning technology concepts and applicationExperience implementing SOA frameworkExperience with enterprise library development and management Experience with API development and managementExperience working in agile scrum environmentAbility to translate data and technical concepts into requirements documents, business cases.",
            "similarity": 0.4534
          },
          {
            "kra_text": "Designs and implements prompt engineering workflows, few-shot examples, chain-of-thought patterns, and structured output parsing for AI feature pipelines.",
            "sentence": "As Lead Data Engineer you will design and implement the next version of our integrated data platform.You will own and drive strategic planning, business analysis and technical implementation of data engineering, tools, and data architecture definition.In addition to managing our portfolio of datasets globally, you will play key role in leading our data engineering community to leverage these datasets effectively.You will also produce documentation and lead training sessions for data engineering solutions.Collaborate with and influence leadership so that your teams work directly impacts company strategy and direction.Develop new techniques and data that will enable answering previously unanswerable and new questions.Develop deep partnerships with engineering and product teams to deliver on major cross-functional measurements, testing.Communicate effectively to all levels of the organization, including executives.Provide success in partnering teams with dramatically varying backgrounds, from the highly technical to the highly creative.",
            "similarity": 0.3654
          }
        ],
        "matched_count": null,
        "matched_skills": null,
        "role_id": 13,
        "score": 0.4094,
        "slug": "ai-engineer",
        "total_count": null
      }
    ],
    "skill_match_roles": [
      {
        "display_name": "Data Engineer",
        "kra_matches": null,
        "matched_count": 5,
        "matched_skills": [
          "Apache Spark",
          "Hadoop",
          "Python",
          "SQL",
          "Scala"
        ],
        "role_id": 2,
        "score": 0.4167,
        "slug": "data-engineer",
        "total_count": 12
      },
      {
        "display_name": "ML Engineer",
        "kra_matches": null,
        "matched_count": 2,
        "matched_skills": [
          "Python",
          "Scala"
        ],
        "role_id": 3,
        "score": 0.1667,
        "slug": "ml-engineer",
        "total_count": 12
      },
      {
        "display_name": "MLOps Engineer",
        "kra_matches": null,
        "matched_count": 2,
        "matched_skills": [
          "Python",
          "Scala"
        ],
        "role_id": 16,
        "score": 0.1667,
        "slug": "ml-ops-engineer",
        "total_count": 12
      },
      {
        "display_name": "Backend Developer",
        "kra_matches": null,
        "matched_count": 2,
        "matched_skills": [
          "NoSQL",
          "Python"
        ],
        "role_id": 1,
        "score": 0.1667,
        "slug": "backend-engineer",
        "total_count": 12
      },
      {
        "display_name": "Cyber Security Engineer",
        "kra_matches": null,
        "matched_count": 1,
        "matched_skills": [
          "Python"
        ],
        "role_id": 5,
        "score": 0.0833,
        "slug": "cybersecurity-engineer",
        "total_count": 12
      }
    ]
  },
  "stage4_decision": {
    "alias_collision_detected": false,
    "case": "A",
    "chosen_role": {
      "display_name": "Data Engineer",
      "kra_matches": null,
      "matched_count": null,
      "matched_skills": null,
      "role_id": 2,
      "score": 1.0,
      "slug": "data-engineer",
      "total_count": null
    },
    "confidence": 1.0,
    "is_new_role": false,
    "llm2_fired": false,
    "llm2_reasoning": null,
    "matched_dimensions": [],
    "matched_kras": [],
    "matched_skills": [],
    "new_role_display_name": null,
    "new_role_slug": null,
    "queued": false,
    "reasoning": "Exact alias hit on data-engineer (1.0) \u2014 no other alias at this confidence; skill_top data-engineer 0.42 does not contradict",
    "sub_role": null
  },
  "stage5_updates": {
    "centroid_n_after": 197,
    "centroid_updated": true,
    "collision_log_id": null,
    "new_kra_attached": null,
    "new_skills_attached": [
      {
        "is_primary": true,
        "queue_id": 10319,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "Data Engineering",
        "status": "pending"
      },
      {
        "is_primary": true,
        "queue_id": 10320,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "Data Architecture",
        "status": "pending"
      },
      {
        "is_primary": true,
        "queue_id": 10321,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "EMR",
        "status": "pending"
      },
      {
        "is_primary": true,
        "queue_id": 10322,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "Unix",
        "status": "pending"
      },
      {
        "is_primary": true,
        "queue_id": 10323,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "Shell",
        "status": "pending"
      },
      {
        "is_primary": false,
        "queue_id": 10324,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "Oozie",
        "status": "pending"
      },
      {
        "is_primary": false,
        "queue_id": 10325,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "Apache Storm",
        "status": "pending"
      },
      {
        "is_primary": false,
        "queue_id": 10326,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "NiFi",
        "status": "pending"
      },
      {
        "is_primary": false,
        "queue_id": 10327,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "D3",
        "status": "pending"
      },
      {
        "is_primary": false,
        "queue_id": 10328,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "Master Data Management",
        "status": "pending"
      },
      {
        "is_primary": false,
        "queue_id": 10329,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "SOA",
        "status": "pending"
      },
      {
        "is_primary": false,
        "queue_id": 10330,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "API Development",
        "status": "pending"
      },
      {
        "is_primary": false,
        "queue_id": 10331,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "MLlib",
        "status": "pending"
      },
      {
        "is_primary": false,
        "queue_id": 10332,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "GraphX",
        "status": "pending"
      },
      {
        "is_primary": false,
        "queue_id": 10333,
        "role_display_name": "Data Engineer",
        "role_slug": "data-engineer",
        "skill_name": "Pandas",
        "status": "pending"
      }
    ],
    "queue_entry_id": null,
    "v3_pipeline_triggered": false,
    "v3_role_slug": null,
    "v3_run_id": null
  }
}
API 2 — extract-details
{
  "alias_matches": [
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 271,
      "existing_alias_text": "SQL",
      "input_term": "SQL",
      "matched_canonical": {
        "category_id": 6,
        "display_name": "SQL",
        "id": 101,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "LANGUAGE",
        "slug": "sql",
        "sub_category_id": 97,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 67,
      "existing_alias_text": "Python",
      "input_term": "Python",
      "matched_canonical": {
        "category_id": 6,
        "display_name": "Python",
        "id": 5,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "LANGUAGE",
        "slug": "python",
        "sub_category_id": 96,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 2004,
      "existing_alias_text": "Apache Spark",
      "input_term": "Apache Spark",
      "matched_canonical": {
        "category_id": 5,
        "display_name": "Apache Spark",
        "id": 1350,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "FRAMEWORK",
        "slug": "apache-spark",
        "sub_category_id": 1021,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 4198,
      "existing_alias_text": "Hive",
      "input_term": "Hive",
      "matched_canonical": {
        "category_id": 3,
        "display_name": "Hive",
        "id": 2754,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "TOOL",
        "slug": "hive",
        "sub_category_id": 2242,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 272,
      "existing_alias_text": "Scala",
      "input_term": "Scala",
      "matched_canonical": {
        "category_id": 6,
        "display_name": "Scala",
        "id": 102,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "LANGUAGE",
        "slug": "scala",
        "sub_category_id": 96,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 2010,
      "existing_alias_text": "Hadoop",
      "input_term": "Hadoop",
      "matched_canonical": {
        "category_id": 5,
        "display_name": "Hadoop",
        "id": 1351,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "FRAMEWORK",
        "slug": "hadoop",
        "sub_category_id": 91,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 1989,
      "existing_alias_text": "NoSQL",
      "input_term": "NoSQL",
      "matched_canonical": {
        "category_id": 2,
        "display_name": "NoSQL",
        "id": 1346,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "CONCEPT",
        "slug": "nosql",
        "sub_category_id": 1019,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 526,
      "existing_alias_text": "Airflow",
      "input_term": "Airflow",
      "matched_canonical": {
        "category_id": 13,
        "display_name": "Airflow",
        "id": 265,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "TOOL",
        "slug": "airflow",
        "sub_category_id": 130,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 319,
      "existing_alias_text": "Spark Streaming",
      "input_term": "Spark Streaming",
      "matched_canonical": {
        "category_id": 5,
        "display_name": "Spark Streaming",
        "id": 121,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "FRAMEWORK",
        "slug": "spark-streaming",
        "sub_category_id": 94,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 359,
      "existing_alias_text": "Tableau",
      "input_term": "Tableau",
      "matched_canonical": {
        "category_id": 9,
        "display_name": "Tableau",
        "id": 150,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "PLATFORM",
        "slug": "tableau",
        "sub_category_id": 111,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 360,
      "existing_alias_text": "Power BI",
      "input_term": "Power BI",
      "matched_canonical": {
        "category_id": 9,
        "display_name": "Power BI",
        "id": 151,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "PLATFORM",
        "slug": "power-bi",
        "sub_category_id": 111,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 1613,
      "existing_alias_text": "Git",
      "input_term": "Git",
      "matched_canonical": {
        "category_id": 13,
        "display_name": "Git",
        "id": 1002,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "TOOL",
        "slug": "git",
        "sub_category_id": 730,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 1826,
      "existing_alias_text": "CI/CD",
      "input_term": "CI/CD",
      "matched_canonical": {
        "category_id": 8,
        "display_name": "CI/CD",
        "id": 1190,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "METHODOLOGY",
        "slug": "ci-cd",
        "sub_category_id": 900,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 2015,
      "existing_alias_text": "Machine Learning",
      "input_term": "Machine Learning",
      "matched_canonical": {
        "category_id": 2,
        "display_name": "Machine Learning",
        "id": 1356,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "CONCEPT",
        "slug": "machine-learning",
        "sub_category_id": 1024,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 868,
      "existing_alias_text": "Agile",
      "input_term": "Agile",
      "matched_canonical": {
        "category_id": 8,
        "display_name": "Agile",
        "id": 520,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "METHODOLOGY",
        "slug": "agile",
        "sub_category_id": 367,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    },
    {
      "alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
      "alias_persisted": false,
      "existing_alias_id": 1851,
      "existing_alias_text": "Scrum",
      "input_term": "Scrum",
      "matched_canonical": {
        "category_id": 8,
        "display_name": "Scrum",
        "id": 1215,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "METHODOLOGY",
        "slug": "scrum",
        "sub_category_id": 921,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "matched_via": "alias"
    }
  ],
  "candidate_roles": [
    {
      "display_name": "Pega Developer",
      "id": 24,
      "rationale": null,
      "role_archetype": null,
      "slug": "pega-developer",
      "source": "db"
    },
    {
      "display_name": "Data Engineer",
      "id": 2,
      "rationale": null,
      "role_archetype": null,
      "slug": "data-engineer",
      "source": "db"
    },
    {
      "display_name": "Cloud Security Engineer",
      "id": 23,
      "rationale": null,
      "role_archetype": null,
      "slug": "cloud-security-engineer",
      "source": "db"
    },
    {
      "display_name": "Backend Developer",
      "id": 1,
      "rationale": null,
      "role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
      "slug": "backend-engineer",
      "source": "db"
    },
    {
      "display_name": "Fullstack Developer",
      "id": 435,
      "rationale": null,
      "role_archetype": "Engineering",
      "slug": "fullstack-developer",
      "source": "db"
    },
    {
      "display_name": "Fullstack Developer",
      "id": 15,
      "rationale": null,
      "role_archetype": null,
      "slug": "full-stack-engineer",
      "source": "db"
    },
    {
      "display_name": "Cyber Security Engineer",
      "id": 5,
      "rationale": null,
      "role_archetype": null,
      "slug": "cybersecurity-engineer",
      "source": "db"
    },
    {
      "display_name": "ML Engineer",
      "id": 3,
      "rationale": null,
      "role_archetype": null,
      "slug": "ml-engineer",
      "source": "db"
    },
    {
      "display_name": "MLOps Engineer",
      "id": 16,
      "rationale": null,
      "role_archetype": null,
      "slug": "ml-ops-engineer",
      "source": "db"
    },
    {
      "display_name": "AR/VR Engineer",
      "id": 8,
      "rationale": null,
      "role_archetype": null,
      "slug": "ar-vr-engineer",
      "source": "db"
    },
    {
      "display_name": "Python Backend Developer",
      "id": 80,
      "rationale": null,
      "role_archetype": "Engineering",
      "slug": "python-backend-developer",
      "source": "db"
    },
    {
      "display_name": "Android Developer",
      "id": 4,
      "rationale": null,
      "role_archetype": null,
      "slug": "android-engineer",
      "source": "db"
    },
    {
      "display_name": "Flutter Developer",
      "id": 74,
      "rationale": null,
      "role_archetype": "Engineering",
      "slug": "flutter-developer",
      "source": "db"
    },
    {
      "display_name": "Hybrid Mobile Developer",
      "id": 11,
      "rationale": null,
      "role_archetype": null,
      "slug": "hybrid-mobile-developer",
      "source": "db"
    },
    {
      "display_name": "Native Mobile Developer",
      "id": 75,
      "rationale": null,
      "role_archetype": "Engineering",
      "slug": "native-mobile-developer",
      "source": "db"
    },
    {
      "display_name": "React Native Developer",
      "id": 73,
      "rationale": null,
      "role_archetype": "Engineering",
      "slug": "react-native-developer",
      "source": "db"
    },
    {
      "display_name": "iOS Developer",
      "id": 6,
      "rationale": null,
      "role_archetype": null,
      "slug": "ios-engineer",
      "source": "db"
    },
    {
      "display_name": "DevOps Engineer",
      "id": 10,
      "rationale": null,
      "role_archetype": null,
      "slug": "devops-engineer",
      "source": "db"
    },
    {
      "display_name": "AI Engineer",
      "id": 13,
      "rationale": null,
      "role_archetype": null,
      "slug": "ai-engineer",
      "source": "db"
    }
  ],
  "chosen_role": {
    "display_name": "Data Engineer",
    "id": 2,
    "rationale": "Exact alias hit on data-engineer (1.0) \u2014 no other alias at this confidence; skill_top data-engineer 0.42 does not contradict",
    "role_archetype": null,
    "slug": "data-engineer",
    "source": "db"
  },
  "dimensions": [
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Pega Programming Languages \u0026 DSLs",
        "id": 267,
        "rationale": "Programming languages and domain-specific languages used in Pega development.",
        "slug": "pega-programming-languages-dsls",
        "source": "db"
      },
      "input_skill": "SQL",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Pega Developer",
          "id": 24,
          "rationale": null,
          "role_archetype": null,
          "slug": "pega-developer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Programming Languages for Data Work",
        "id": 21,
        "rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
        "slug": "programming-languages-for-data-work",
        "source": "db"
      },
      "input_skill": "SQL",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Data Engineer",
          "id": 2,
          "rationale": null,
          "role_archetype": null,
          "slug": "data-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Cloud Security Scripting \u0026 DSL Languages",
        "id": 248,
        "rationale": "Proficiency in programming and domain-specific languages used to automate and script cloud security controls.",
        "slug": "cloud-security-scripting-dsl-languages",
        "source": "db"
      },
      "input_skill": "Python",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Cloud Security Engineer",
          "id": 23,
          "rationale": null,
          "role_archetype": null,
          "slug": "cloud-security-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Programming Languages",
        "id": 1,
        "rationale": "Primary implementation languages used to build client and server feature code. Full stack engineers need enough fluency to move across layers and implement product behavior end to end.",
        "slug": "programming-languages",
        "source": "db"
      },
      "input_skill": "Python",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Backend Developer",
          "id": 1,
          "rationale": null,
          "role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
          "slug": "backend-engineer",
          "source": "db"
        },
        {
          "display_name": "Fullstack Developer",
          "id": 435,
          "rationale": null,
          "role_archetype": "Engineering",
          "slug": "fullstack-developer",
          "source": "db"
        },
        {
          "display_name": "Fullstack Developer",
          "id": 15,
          "rationale": null,
          "role_archetype": null,
          "slug": "full-stack-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Programming Languages and Scripting",
        "id": 59,
        "rationale": "Languages used to write security automation, analysis scripts, detection logic, and remediation helpers. This is the primary implementation surface for a cybersecurity engineer across tooling and response workflows.",
        "slug": "programming-languages-and-scripting",
        "source": "db"
      },
      "input_skill": "Python",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Cyber Security Engineer",
          "id": 5,
          "rationale": null,
          "role_archetype": null,
          "slug": "cybersecurity-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Programming Languages for Data Work",
        "id": 21,
        "rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
        "slug": "programming-languages-for-data-work",
        "source": "db"
      },
      "input_skill": "Python",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Data Engineer",
          "id": 2,
          "rationale": null,
          "role_archetype": null,
          "slug": "data-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Programming Languages for ML Systems",
        "id": 39,
        "rationale": "Languages used to build training code, inference services, evaluation jobs, and ML glue code. This is the primary implementation surface for ML engineers across experimentation and productionization.",
        "slug": "programming-languages-for-ml-systems",
        "source": "db"
      },
      "input_skill": "Python",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "ML Engineer",
          "id": 3,
          "rationale": null,
          "role_archetype": null,
          "slug": "ml-engineer",
          "source": "db"
        },
        {
          "display_name": "MLOps Engineer",
          "id": 16,
          "rationale": null,
          "role_archetype": null,
          "slug": "ml-ops-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Programming Languages for XR",
        "id": 97,
        "rationale": "Primary implementation languages used to build immersive client features, interaction logic, and device-specific runtime behavior. This is the core coding surface for AR/VR experiences.",
        "slug": "programming-languages-for-xr",
        "source": "db"
      },
      "input_skill": "Python",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "AR/VR Engineer",
          "id": 8,
          "rationale": null,
          "role_archetype": null,
          "slug": "ar-vr-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Python Programming",
        "id": 290,
        "rationale": "Core Python language skills used to implement backend business logic, request handlers, integrations, and service internals. This is the primary coding surface for the role.",
        "slug": "python-programming",
        "source": "db"
      },
      "input_skill": "Python",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Python Backend Developer",
          "id": 80,
          "rationale": null,
          "role_archetype": "Engineering",
          "slug": "python-backend-developer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "ETL and ELT Tooling",
        "id": 24,
        "rationale": "Packaged tools for extracting, loading, and transforming data across systems. This dimension covers connector-based ingestion, transformation frameworks, and managed integration products.",
        "slug": "etl-and-elt-tooling",
        "source": "db"
      },
      "input_skill": "Apache Spark",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Data Engineer",
          "id": 2,
          "rationale": null,
          "role_archetype": null,
          "slug": "data-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Local Persistence and Offline Behavior",
        "id": 85,
        "rationale": "On-device storage used for caching, offline support, and durable client state. This cluster is coherent because iOS apps often need to preserve user progress and data when connectivity is limited.",
        "slug": "local-persistence-and-offline-behavior",
        "source": "db"
      },
      "input_skill": "Hive",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Android Developer",
          "id": 4,
          "rationale": null,
          "role_archetype": null,
          "slug": "android-engineer",
          "source": "db"
        },
        {
          "display_name": "Flutter Developer",
          "id": 74,
          "rationale": null,
          "role_archetype": "Engineering",
          "slug": "flutter-developer",
          "source": "db"
        },
        {
          "display_name": "Hybrid Mobile Developer",
          "id": 11,
          "rationale": null,
          "role_archetype": null,
          "slug": "hybrid-mobile-developer",
          "source": "db"
        },
        {
          "display_name": "Native Mobile Developer",
          "id": 75,
          "rationale": null,
          "role_archetype": "Engineering",
          "slug": "native-mobile-developer",
          "source": "db"
        },
        {
          "display_name": "React Native Developer",
          "id": 73,
          "rationale": null,
          "role_archetype": "Engineering",
          "slug": "react-native-developer",
          "source": "db"
        },
        {
          "display_name": "iOS Developer",
          "id": 6,
          "rationale": null,
          "role_archetype": null,
          "slug": "ios-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Programming Languages for Data Work",
        "id": 21,
        "rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
        "slug": "programming-languages-for-data-work",
        "source": "db"
      },
      "input_skill": "Scala",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Data Engineer",
          "id": 2,
          "rationale": null,
          "role_archetype": null,
          "slug": "data-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Programming Languages for ML Systems",
        "id": 39,
        "rationale": "Languages used to build training code, inference services, evaluation jobs, and ML glue code. This is the primary implementation surface for ML engineers across experimentation and productionization.",
        "slug": "programming-languages-for-ml-systems",
        "source": "db"
      },
      "input_skill": "Scala",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "ML Engineer",
          "id": 3,
          "rationale": null,
          "role_archetype": null,
          "slug": "ml-engineer",
          "source": "db"
        },
        {
          "display_name": "MLOps Engineer",
          "id": 16,
          "rationale": null,
          "role_archetype": null,
          "slug": "ml-ops-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "ETL and ELT Tooling",
        "id": 24,
        "rationale": "Packaged tools for extracting, loading, and transforming data across systems. This dimension covers connector-based ingestion, transformation frameworks, and managed integration products.",
        "slug": "etl-and-elt-tooling",
        "source": "db"
      },
      "input_skill": "Hadoop",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Data Engineer",
          "id": 2,
          "rationale": null,
          "role_archetype": null,
          "slug": "data-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "NoSQL Databases",
        "id": 19,
        "rationale": "Models and manages data using non-relational database systems.",
        "slug": "nosql-databases",
        "source": "db"
      },
      "input_skill": "NoSQL",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Backend Developer",
          "id": 1,
          "rationale": null,
          "role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
          "slug": "backend-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Workflow Orchestration for ML Pipelines",
        "id": 54,
        "rationale": "Workflow engines used to coordinate training, evaluation, deployment, and retraining jobs. This cluster covers dependencies, retries, scheduling, and pipeline composition for ML lifecycle automation.",
        "slug": "workflow-orchestration-for-ml-pipelines",
        "source": "db"
      },
      "input_skill": "Airflow",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "ML Engineer",
          "id": 3,
          "rationale": null,
          "role_archetype": null,
          "slug": "ml-engineer",
          "source": "db"
        },
        {
          "display_name": "MLOps Engineer",
          "id": 16,
          "rationale": null,
          "role_archetype": null,
          "slug": "ml-ops-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "Stream Processing Systems",
        "id": 25,
        "rationale": "Technologies for processing event streams and near-real-time data flows. This includes stream transformations, windowing, stateful processing, and stream-to-warehouse delivery patterns.",
        "slug": "stream-processing-systems",
        "source": "db"
      },
      "input_skill": "Spark Streaming",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Data Engineer",
          "id": 2,
          "rationale": null,
          "role_archetype": null,
          "slug": "data-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "BI and Visualization Tools",
        "id": 31,
        "rationale": "Tools used to expose curated data to analysts and business users through dashboards, reports, and semantic exploration. Data engineers support these tools by shaping reliable datasets and performant models.",
        "slug": "bi-and-visualization-tools",
        "source": "db"
      },
      "input_skill": "Tableau",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Data Engineer",
          "id": 2,
          "rationale": null,
          "role_archetype": null,
          "slug": "data-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "BI and Visualization Tools",
        "id": 31,
        "rationale": "Tools used to expose curated data to analysts and business users through dashboards, reports, and semantic exploration. Data engineers support these tools by shaping reliable datasets and performant models.",
        "slug": "bi-and-visualization-tools",
        "source": "db"
      },
      "input_skill": "Power BI",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "Data Engineer",
          "id": 2,
          "rationale": null,
          "role_archetype": null,
          "slug": "data-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "React Frontend Development",
        "id": 96,
        "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
        "slug": "d_init_01",
        "source": "db"
      },
      "input_skill": "Git",
      "llm_role": null,
      "roles_from_db": []
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "CI/CD Pipeline Platforms",
        "id": 150,
        "rationale": "Systems used to define, run, and maintain automated build and deployment workflows. This cluster is coherent because the role owns delivery automation end to end, including pipeline reliability and promotion logic.",
        "slug": "ci-cd-pipeline-platforms",
        "source": "db"
      },
      "input_skill": "CI/CD",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "DevOps Engineer",
          "id": 10,
          "rationale": null,
          "role_archetype": null,
          "slug": "devops-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "CI/CD for Machine Learning",
        "id": 56,
        "rationale": "Tools and platforms for automating ML model integration, testing, and deployment pipelines.",
        "slug": "ci-cd-for-machine-learning",
        "source": "db"
      },
      "input_skill": "CI/CD",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "ML Engineer",
          "id": 3,
          "rationale": null,
          "role_archetype": null,
          "slug": "ml-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "AI Governance and Model Security",
        "id": 50,
        "rationale": "Controls and documentation used to make models safer, auditable, and compliant. ML engineers use this to manage model risk, supply chain integrity, and governance requirements.",
        "slug": "ai-governance-and-model-security",
        "source": "db"
      },
      "input_skill": "Machine Learning",
      "llm_role": null,
      "roles_from_db": [
        {
          "display_name": "AI Engineer",
          "id": 13,
          "rationale": null,
          "role_archetype": null,
          "slug": "ai-engineer",
          "source": "db"
        },
        {
          "display_name": "ML Engineer",
          "id": 3,
          "rationale": null,
          "role_archetype": null,
          "slug": "ml-engineer",
          "source": "db"
        },
        {
          "display_name": "MLOps Engineer",
          "id": 16,
          "rationale": null,
          "role_archetype": null,
          "slug": "ml-ops-engineer",
          "source": "db"
        }
      ]
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "React Frontend Development",
        "id": 96,
        "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
        "slug": "d_init_01",
        "source": "db"
      },
      "input_skill": "Machine Learning",
      "llm_role": null,
      "roles_from_db": []
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "React Frontend Development",
        "id": 96,
        "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
        "slug": "d_init_01",
        "source": "db"
      },
      "input_skill": "Agile",
      "llm_role": null,
      "roles_from_db": []
    },
    {
      "dimension": {
        "difficulty_hint": "well_known",
        "display_name": "React Frontend Development",
        "id": 96,
        "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
        "slug": "d_init_01",
        "source": "db"
      },
      "input_skill": "Scrum",
      "llm_role": null,
      "roles_from_db": []
    }
  ],
  "input_final_skills": [
    "Data Engineering",
    "Data Architecture",
    "SQL",
    "Python",
    "Apache Spark",
    "Hive",
    "Scala",
    "Hadoop",
    "EMR",
    "NoSQL",
    "Unix",
    "Shell",
    "Oozie",
    "Airflow",
    "Apache Storm",
    "Spark Streaming",
    "NiFi",
    "Tableau",
    "Power BI",
    "D3",
    "Git",
    "CI/CD",
    "Master Data Management",
    "Machine Learning",
    "SOA",
    "API Development",
    "Agile",
    "Scrum",
    "MLlib",
    "GraphX",
    "Pandas"
  ],
  "input_llm_skills": [
    "Data Engineering",
    "Data Architecture",
    "SQL",
    "Python",
    "Apache Spark",
    "Hive",
    "Scala",
    "Hadoop",
    "EMR",
    "NoSQL",
    "Unix",
    "Shell",
    "Oozie",
    "Airflow",
    "Apache Storm",
    "Spark Streaming",
    "NiFi",
    "Tableau",
    "Power BI",
    "D3",
    "Git",
    "CI/CD",
    "Master Data Management",
    "Machine Learning",
    "SOA",
    "API Development",
    "Agile",
    "Scrum",
    "MLlib",
    "GraphX",
    "Pandas"
  ],
  "new_aliases_persisted": 0,
  "run_id": "459d8d5b-8e49-4c9b-a801-9389a9c4c4ae",
  "skills_detail": [
    {
      "aliases_in_db": [],
      "canonical": null,
      "dimensions": [],
      "input_skill": "Data Engineering",
      "matched_via": null,
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": {
        "derived": {
          "category": "Data Engineering Tools",
          "skill_nature": "PRACTICE",
          "sub_category": "general",
          "typical_lifespan": "MULTI_YEAR",
          "version_strategy": "UNVERSIONED",
          "volatility": "MEDIUM"
        },
        "enrichment": null,
        "keep_log": [],
        "locked_dimensions": [],
        "merge_log": [],
        "placed": null,
        "relationships": null,
        "skill_id": "data-engineering",
        "split_log": [],
        "typed": null,
        "warnings": []
      },
      "source_tag": "llm",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [],
      "canonical": null,
      "dimensions": [],
      "input_skill": "Data Architecture",
      "matched_via": null,
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": {
        "derived": {
          "category": "Data Engineering Tools",
          "skill_nature": "CONCEPT",
          "sub_category": "general",
          "typical_lifespan": "MULTI_YEAR",
          "version_strategy": "UNVERSIONED",
          "volatility": "MEDIUM"
        },
        "enrichment": null,
        "keep_log": [],
        "locked_dimensions": [],
        "merge_log": [],
        "placed": null,
        "relationships": null,
        "skill_id": "data-architecture",
        "split_log": [],
        "typed": null,
        "warnings": []
      },
      "source_tag": "llm",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "SQL",
          "alias_type": "CANONICAL",
          "id": 271,
          "is_primary": true,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 6,
        "display_name": "SQL",
        "id": 101,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "LANGUAGE",
        "slug": "sql",
        "sub_category_id": 97,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Pega Programming Languages \u0026 DSLs",
            "id": 267,
            "rationale": "Programming languages and domain-specific languages used in Pega development.",
            "slug": "pega-programming-languages-dsls",
            "source": "db"
          },
          "input_skill": "SQL",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Pega Developer",
              "id": 24,
              "rationale": null,
              "role_archetype": null,
              "slug": "pega-developer",
              "source": "db"
            }
          ]
        },
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Programming Languages for Data Work",
            "id": 21,
            "rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
            "slug": "programming-languages-for-data-work",
            "source": "db"
          },
          "input_skill": "SQL",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Data Engineer",
              "id": 2,
              "rationale": null,
              "role_archetype": null,
              "slug": "data-engineer",
              "source": "db"
            }
          ]
        }
      ],
      "input_skill": "SQL",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "Python",
          "alias_type": "CANONICAL",
          "id": 67,
          "is_primary": true,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "Python 2",
          "alias_type": "VERSION",
          "id": 72,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "Python 2.x",
          "alias_type": "VERSION",
          "id": 74,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "Python 3",
          "alias_type": "VERSION",
          "id": 73,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "Python 3.10",
          "alias_type": "VERSION",
          "id": 76,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "Python 3.11",
          "alias_type": "VERSION",
          "id": 77,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "Python 3.12",
          "alias_type": "VERSION",
          "id": 78,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "Python 3.x",
          "alias_type": "VERSION",
          "id": 75,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "py",
          "alias_type": "VERSION",
          "id": 2183,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "py2",
          "alias_type": "VERSION",
          "id": 68,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "py3",
          "alias_type": "VERSION",
          "id": 69,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "python 3",
          "alias_type": "VERSION",
          "id": 2186,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "python 3.x",
          "alias_type": "VERSION",
          "id": 2849,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "python2",
          "alias_type": "VERSION",
          "id": 70,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "python3",
          "alias_type": "VERSION",
          "id": 71,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "python3.x",
          "alias_type": "VERSION",
          "id": 2848,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 6,
        "display_name": "Python",
        "id": 5,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "LANGUAGE",
        "slug": "python",
        "sub_category_id": 96,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Cloud Security Scripting \u0026 DSL Languages",
            "id": 248,
            "rationale": "Proficiency in programming and domain-specific languages used to automate and script cloud security controls.",
            "slug": "cloud-security-scripting-dsl-languages",
            "source": "db"
          },
          "input_skill": "Python",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Cloud Security Engineer",
              "id": 23,
              "rationale": null,
              "role_archetype": null,
              "slug": "cloud-security-engineer",
              "source": "db"
            }
          ]
        },
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Programming Languages",
            "id": 1,
            "rationale": "Primary implementation languages used to build client and server feature code. Full stack engineers need enough fluency to move across layers and implement product behavior end to end.",
            "slug": "programming-languages",
            "source": "db"
          },
          "input_skill": "Python",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Backend Developer",
              "id": 1,
              "rationale": null,
              "role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
              "slug": "backend-engineer",
              "source": "db"
            },
            {
              "display_name": "Fullstack Developer",
              "id": 435,
              "rationale": null,
              "role_archetype": "Engineering",
              "slug": "fullstack-developer",
              "source": "db"
            },
            {
              "display_name": "Fullstack Developer",
              "id": 15,
              "rationale": null,
              "role_archetype": null,
              "slug": "full-stack-engineer",
              "source": "db"
            }
          ]
        },
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Programming Languages and Scripting",
            "id": 59,
            "rationale": "Languages used to write security automation, analysis scripts, detection logic, and remediation helpers. This is the primary implementation surface for a cybersecurity engineer across tooling and response workflows.",
            "slug": "programming-languages-and-scripting",
            "source": "db"
          },
          "input_skill": "Python",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Cyber Security Engineer",
              "id": 5,
              "rationale": null,
              "role_archetype": null,
              "slug": "cybersecurity-engineer",
              "source": "db"
            }
          ]
        },
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Programming Languages for Data Work",
            "id": 21,
            "rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
            "slug": "programming-languages-for-data-work",
            "source": "db"
          },
          "input_skill": "Python",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Data Engineer",
              "id": 2,
              "rationale": null,
              "role_archetype": null,
              "slug": "data-engineer",
              "source": "db"
            }
          ]
        },
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Programming Languages for ML Systems",
            "id": 39,
            "rationale": "Languages used to build training code, inference services, evaluation jobs, and ML glue code. This is the primary implementation surface for ML engineers across experimentation and productionization.",
            "slug": "programming-languages-for-ml-systems",
            "source": "db"
          },
          "input_skill": "Python",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "ML Engineer",
              "id": 3,
              "rationale": null,
              "role_archetype": null,
              "slug": "ml-engineer",
              "source": "db"
            },
            {
              "display_name": "MLOps Engineer",
              "id": 16,
              "rationale": null,
              "role_archetype": null,
              "slug": "ml-ops-engineer",
              "source": "db"
            }
          ]
        },
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Programming Languages for XR",
            "id": 97,
            "rationale": "Primary implementation languages used to build immersive client features, interaction logic, and device-specific runtime behavior. This is the core coding surface for AR/VR experiences.",
            "slug": "programming-languages-for-xr",
            "source": "db"
          },
          "input_skill": "Python",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "AR/VR Engineer",
              "id": 8,
              "rationale": null,
              "role_archetype": null,
              "slug": "ar-vr-engineer",
              "source": "db"
            }
          ]
        },
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Python Programming",
            "id": 290,
            "rationale": "Core Python language skills used to implement backend business logic, request handlers, integrations, and service internals. This is the primary coding surface for the role.",
            "slug": "python-programming",
            "source": "db"
          },
          "input_skill": "Python",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Python Backend Developer",
              "id": 80,
              "rationale": null,
              "role_archetype": "Engineering",
              "slug": "python-backend-developer",
              "source": "db"
            }
          ]
        }
      ],
      "input_skill": "Python",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "Apache Spark",
          "alias_type": "CANONICAL",
          "id": 2004,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "apache spark 3",
          "alias_type": "VERSION",
          "id": 2006,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "spark",
          "alias_type": "VERSION",
          "id": 2510,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "spark 3",
          "alias_type": "VERSION",
          "id": 2007,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "spark 3.x",
          "alias_type": "VERSION",
          "id": 2009,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "spark3",
          "alias_type": "VERSION",
          "id": 2008,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 5,
        "display_name": "Apache Spark",
        "id": 1350,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "FRAMEWORK",
        "slug": "apache-spark",
        "sub_category_id": 1021,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "ETL and ELT Tooling",
            "id": 24,
            "rationale": "Packaged tools for extracting, loading, and transforming data across systems. This dimension covers connector-based ingestion, transformation frameworks, and managed integration products.",
            "slug": "etl-and-elt-tooling",
            "source": "db"
          },
          "input_skill": "Apache Spark",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Data Engineer",
              "id": 2,
              "rationale": null,
              "role_archetype": null,
              "slug": "data-engineer",
              "source": "db"
            }
          ]
        }
      ],
      "input_skill": "Apache Spark",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "Hive",
          "alias_type": "CANONICAL",
          "id": 4198,
          "is_primary": true,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 3,
        "display_name": "Hive",
        "id": 2754,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "TOOL",
        "slug": "hive",
        "sub_category_id": 2242,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Local Persistence and Offline Behavior",
            "id": 85,
            "rationale": "On-device storage used for caching, offline support, and durable client state. This cluster is coherent because iOS apps often need to preserve user progress and data when connectivity is limited.",
            "slug": "local-persistence-and-offline-behavior",
            "source": "db"
          },
          "input_skill": "Hive",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Android Developer",
              "id": 4,
              "rationale": null,
              "role_archetype": null,
              "slug": "android-engineer",
              "source": "db"
            },
            {
              "display_name": "Flutter Developer",
              "id": 74,
              "rationale": null,
              "role_archetype": "Engineering",
              "slug": "flutter-developer",
              "source": "db"
            },
            {
              "display_name": "Hybrid Mobile Developer",
              "id": 11,
              "rationale": null,
              "role_archetype": null,
              "slug": "hybrid-mobile-developer",
              "source": "db"
            },
            {
              "display_name": "Native Mobile Developer",
              "id": 75,
              "rationale": null,
              "role_archetype": "Engineering",
              "slug": "native-mobile-developer",
              "source": "db"
            },
            {
              "display_name": "React Native Developer",
              "id": 73,
              "rationale": null,
              "role_archetype": "Engineering",
              "slug": "react-native-developer",
              "source": "db"
            },
            {
              "display_name": "iOS Developer",
              "id": 6,
              "rationale": null,
              "role_archetype": null,
              "slug": "ios-engineer",
              "source": "db"
            }
          ]
        }
      ],
      "input_skill": "Hive",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "Scala",
          "alias_type": "CANONICAL",
          "id": 272,
          "is_primary": true,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 6,
        "display_name": "Scala",
        "id": 102,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "LANGUAGE",
        "slug": "scala",
        "sub_category_id": 96,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Programming Languages for Data Work",
            "id": 21,
            "rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
            "slug": "programming-languages-for-data-work",
            "source": "db"
          },
          "input_skill": "Scala",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Data Engineer",
              "id": 2,
              "rationale": null,
              "role_archetype": null,
              "slug": "data-engineer",
              "source": "db"
            }
          ]
        },
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Programming Languages for ML Systems",
            "id": 39,
            "rationale": "Languages used to build training code, inference services, evaluation jobs, and ML glue code. This is the primary implementation surface for ML engineers across experimentation and productionization.",
            "slug": "programming-languages-for-ml-systems",
            "source": "db"
          },
          "input_skill": "Scala",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "ML Engineer",
              "id": 3,
              "rationale": null,
              "role_archetype": null,
              "slug": "ml-engineer",
              "source": "db"
            },
            {
              "display_name": "MLOps Engineer",
              "id": 16,
              "rationale": null,
              "role_archetype": null,
              "slug": "ml-ops-engineer",
              "source": "db"
            }
          ]
        }
      ],
      "input_skill": "Scala",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "Hadoop",
          "alias_type": "CANONICAL",
          "id": 2010,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 5,
        "display_name": "Hadoop",
        "id": 1351,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "FRAMEWORK",
        "slug": "hadoop",
        "sub_category_id": 91,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "ETL and ELT Tooling",
            "id": 24,
            "rationale": "Packaged tools for extracting, loading, and transforming data across systems. This dimension covers connector-based ingestion, transformation frameworks, and managed integration products.",
            "slug": "etl-and-elt-tooling",
            "source": "db"
          },
          "input_skill": "Hadoop",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Data Engineer",
              "id": 2,
              "rationale": null,
              "role_archetype": null,
              "slug": "data-engineer",
              "source": "db"
            }
          ]
        }
      ],
      "input_skill": "Hadoop",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [],
      "canonical": null,
      "dimensions": [],
      "input_skill": "EMR",
      "matched_via": null,
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": {
        "derived": {
          "category": "Cloud Platforms",
          "skill_nature": "PLATFORM",
          "sub_category": "general",
          "typical_lifespan": "MULTI_YEAR",
          "version_strategy": "UNVERSIONED",
          "volatility": "MEDIUM"
        },
        "enrichment": null,
        "keep_log": [],
        "locked_dimensions": [],
        "merge_log": [],
        "placed": null,
        "relationships": null,
        "skill_id": "emr",
        "split_log": [],
        "typed": null,
        "warnings": []
      },
      "source_tag": "llm",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "NoSQL",
          "alias_type": "CANONICAL",
          "id": 1989,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 2,
        "display_name": "NoSQL",
        "id": 1346,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "CONCEPT",
        "slug": "nosql",
        "sub_category_id": 1019,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "NoSQL Databases",
            "id": 19,
            "rationale": "Models and manages data using non-relational database systems.",
            "slug": "nosql-databases",
            "source": "db"
          },
          "input_skill": "NoSQL",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Backend Developer",
              "id": 1,
              "rationale": null,
              "role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
              "slug": "backend-engineer",
              "source": "db"
            }
          ]
        }
      ],
      "input_skill": "NoSQL",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [],
      "canonical": null,
      "dimensions": [],
      "input_skill": "Unix",
      "matched_via": null,
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": {
        "derived": {
          "category": "Operating Systems",
          "skill_nature": "TOOL",
          "sub_category": "general",
          "typical_lifespan": "EVERGREEN",
          "version_strategy": "UNVERSIONED",
          "volatility": "STABLE"
        },
        "enrichment": null,
        "keep_log": [],
        "locked_dimensions": [],
        "merge_log": [],
        "placed": null,
        "relationships": null,
        "skill_id": "unix",
        "split_log": [],
        "typed": null,
        "warnings": []
      },
      "source_tag": "llm",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [],
      "canonical": null,
      "dimensions": [],
      "input_skill": "Shell",
      "matched_via": null,
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": {
        "derived": {
          "category": "Scripting Languages",
          "skill_nature": "LANGUAGE",
          "sub_category": "general",
          "typical_lifespan": "EVERGREEN",
          "version_strategy": "UNVERSIONED",
          "volatility": "STABLE"
        },
        "enrichment": null,
        "keep_log": [],
        "locked_dimensions": [],
        "merge_log": [],
        "placed": null,
        "relationships": null,
        "skill_id": "shell",
        "split_log": [],
        "typed": null,
        "warnings": []
      },
      "source_tag": "llm",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [],
      "canonical": null,
      "dimensions": [],
      "input_skill": "Oozie",
      "matched_via": null,
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": {
        "derived": {
          "category": "Data Engineering Tools",
          "skill_nature": "TOOL",
          "sub_category": "general",
          "typical_lifespan": "SHORT_LIVED",
          "version_strategy": "VERSIONED",
          "volatility": "FAST"
        },
        "enrichment": null,
        "keep_log": [],
        "locked_dimensions": [],
        "merge_log": [],
        "placed": null,
        "relationships": null,
        "skill_id": "oozie",
        "split_log": [],
        "typed": null,
        "warnings": []
      },
      "source_tag": "llm",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "Airflow",
          "alias_type": "CANONICAL",
          "id": 526,
          "is_primary": true,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "airflow 2",
          "alias_type": "VERSION",
          "id": 2477,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "airflow-2",
          "alias_type": "VERSION",
          "id": 2478,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "airflow2",
          "alias_type": "VERSION",
          "id": 2476,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "airflow2.x",
          "alias_type": "VERSION",
          "id": 2479,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "apache airflow 2",
          "alias_type": "VERSION",
          "id": 2480,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 13,
        "display_name": "Airflow",
        "id": 265,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "TOOL",
        "slug": "airflow",
        "sub_category_id": 130,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Workflow Orchestration for ML Pipelines",
            "id": 54,
            "rationale": "Workflow engines used to coordinate training, evaluation, deployment, and retraining jobs. This cluster covers dependencies, retries, scheduling, and pipeline composition for ML lifecycle automation.",
            "slug": "workflow-orchestration-for-ml-pipelines",
            "source": "db"
          },
          "input_skill": "Airflow",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "ML Engineer",
              "id": 3,
              "rationale": null,
              "role_archetype": null,
              "slug": "ml-engineer",
              "source": "db"
            },
            {
              "display_name": "MLOps Engineer",
              "id": 16,
              "rationale": null,
              "role_archetype": null,
              "slug": "ml-ops-engineer",
              "source": "db"
            }
          ]
        }
      ],
      "input_skill": "Airflow",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [],
      "canonical": null,
      "dimensions": [],
      "input_skill": "Apache Storm",
      "matched_via": null,
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": {
        "derived": {
          "category": "Data Engineering Tools",
          "skill_nature": "TOOL",
          "sub_category": "general",
          "typical_lifespan": "MULTI_YEAR",
          "version_strategy": "UNVERSIONED",
          "volatility": "MEDIUM"
        },
        "enrichment": null,
        "keep_log": [],
        "locked_dimensions": [],
        "merge_log": [],
        "placed": null,
        "relationships": null,
        "skill_id": "apache-storm",
        "split_log": [],
        "typed": null,
        "warnings": []
      },
      "source_tag": "llm",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "DStreams",
          "alias_type": "VERSION",
          "id": 320,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "Spark 2.x",
          "alias_type": "VERSION",
          "id": 321,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "Spark 3.x",
          "alias_type": "VERSION",
          "id": 322,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "Spark Streaming",
          "alias_type": "VERSION",
          "id": 319,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "Spark Structured Streaming",
          "alias_type": "VERSION",
          "id": 325,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        },
        {
          "alias_text": "Structured Streaming",
          "alias_type": "VERSION",
          "id": 324,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 5,
        "display_name": "Spark Streaming",
        "id": 121,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "FRAMEWORK",
        "slug": "spark-streaming",
        "sub_category_id": 94,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "Stream Processing Systems",
            "id": 25,
            "rationale": "Technologies for processing event streams and near-real-time data flows. This includes stream transformations, windowing, stateful processing, and stream-to-warehouse delivery patterns.",
            "slug": "stream-processing-systems",
            "source": "db"
          },
          "input_skill": "Spark Streaming",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Data Engineer",
              "id": 2,
              "rationale": null,
              "role_archetype": null,
              "slug": "data-engineer",
              "source": "db"
            }
          ]
        }
      ],
      "input_skill": "Spark Streaming",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [],
      "canonical": null,
      "dimensions": [],
      "input_skill": "NiFi",
      "matched_via": null,
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": {
        "derived": {
          "category": "Data Engineering Tools",
          "skill_nature": "TOOL",
          "sub_category": "general",
          "typical_lifespan": "MULTI_YEAR",
          "version_strategy": "UNVERSIONED",
          "volatility": "MEDIUM"
        },
        "enrichment": null,
        "keep_log": [],
        "locked_dimensions": [],
        "merge_log": [],
        "placed": null,
        "relationships": null,
        "skill_id": "nifi",
        "split_log": [],
        "typed": null,
        "warnings": []
      },
      "source_tag": "llm",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "Tableau",
          "alias_type": "CANONICAL",
          "id": 359,
          "is_primary": true,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 9,
        "display_name": "Tableau",
        "id": 150,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "PLATFORM",
        "slug": "tableau",
        "sub_category_id": 111,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "BI and Visualization Tools",
            "id": 31,
            "rationale": "Tools used to expose curated data to analysts and business users through dashboards, reports, and semantic exploration. Data engineers support these tools by shaping reliable datasets and performant models.",
            "slug": "bi-and-visualization-tools",
            "source": "db"
          },
          "input_skill": "Tableau",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Data Engineer",
              "id": 2,
              "rationale": null,
              "role_archetype": null,
              "slug": "data-engineer",
              "source": "db"
            }
          ]
        }
      ],
      "input_skill": "Tableau",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "Power BI",
          "alias_type": "CANONICAL",
          "id": 360,
          "is_primary": true,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 9,
        "display_name": "Power BI",
        "id": 151,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "PLATFORM",
        "slug": "power-bi",
        "sub_category_id": 111,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "BI and Visualization Tools",
            "id": 31,
            "rationale": "Tools used to expose curated data to analysts and business users through dashboards, reports, and semantic exploration. Data engineers support these tools by shaping reliable datasets and performant models.",
            "slug": "bi-and-visualization-tools",
            "source": "db"
          },
          "input_skill": "Power BI",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "Data Engineer",
              "id": 2,
              "rationale": null,
              "role_archetype": null,
              "slug": "data-engineer",
              "source": "db"
            }
          ]
        }
      ],
      "input_skill": "Power BI",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [],
      "canonical": null,
      "dimensions": [],
      "input_skill": "D3",
      "matched_via": null,
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": {
        "derived": {
          "category": "Data Visualization Tools",
          "skill_nature": "TOOL",
          "sub_category": "general",
          "typical_lifespan": "MULTI_YEAR",
          "version_strategy": "UNVERSIONED",
          "volatility": "MEDIUM"
        },
        "enrichment": null,
        "keep_log": [],
        "locked_dimensions": [],
        "merge_log": [],
        "placed": null,
        "relationships": null,
        "skill_id": "d3",
        "split_log": [],
        "typed": null,
        "warnings": []
      },
      "source_tag": "llm",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "Git",
          "alias_type": "CANONICAL",
          "id": 1613,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 13,
        "display_name": "Git",
        "id": 1002,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "TOOL",
        "slug": "git",
        "sub_category_id": 730,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "React Frontend Development",
            "id": 96,
            "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
            "slug": "d_init_01",
            "source": "db"
          },
          "input_skill": "Git",
          "llm_role": null,
          "roles_from_db": []
        }
      ],
      "input_skill": "Git",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "CI/CD",
          "alias_type": "CANONICAL",
          "id": 1826,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 8,
        "display_name": "CI/CD",
        "id": 1190,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "METHODOLOGY",
        "slug": "ci-cd",
        "sub_category_id": 900,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "CI/CD Pipeline Platforms",
            "id": 150,
            "rationale": "Systems used to define, run, and maintain automated build and deployment workflows. This cluster is coherent because the role owns delivery automation end to end, including pipeline reliability and promotion logic.",
            "slug": "ci-cd-pipeline-platforms",
            "source": "db"
          },
          "input_skill": "CI/CD",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "DevOps Engineer",
              "id": 10,
              "rationale": null,
              "role_archetype": null,
              "slug": "devops-engineer",
              "source": "db"
            }
          ]
        },
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "CI/CD for Machine Learning",
            "id": 56,
            "rationale": "Tools and platforms for automating ML model integration, testing, and deployment pipelines.",
            "slug": "ci-cd-for-machine-learning",
            "source": "db"
          },
          "input_skill": "CI/CD",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "ML Engineer",
              "id": 3,
              "rationale": null,
              "role_archetype": null,
              "slug": "ml-engineer",
              "source": "db"
            }
          ]
        }
      ],
      "input_skill": "CI/CD",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [],
      "canonical": null,
      "dimensions": [],
      "input_skill": "Master Data Management",
      "matched_via": null,
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": {
        "derived": {
          "category": "Data Management",
          "skill_nature": "CONCEPT",
          "sub_category": "general",
          "typical_lifespan": "MULTI_YEAR",
          "version_strategy": "UNVERSIONED",
          "volatility": "MEDIUM"
        },
        "enrichment": null,
        "keep_log": [],
        "locked_dimensions": [],
        "merge_log": [],
        "placed": null,
        "relationships": null,
        "skill_id": "master-data-management",
        "split_log": [],
        "typed": null,
        "warnings": []
      },
      "source_tag": "llm",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "Machine Learning",
          "alias_type": "CANONICAL",
          "id": 2015,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 2,
        "display_name": "Machine Learning",
        "id": 1356,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "CONCEPT",
        "slug": "machine-learning",
        "sub_category_id": 1024,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "AI Governance and Model Security",
            "id": 50,
            "rationale": "Controls and documentation used to make models safer, auditable, and compliant. ML engineers use this to manage model risk, supply chain integrity, and governance requirements.",
            "slug": "ai-governance-and-model-security",
            "source": "db"
          },
          "input_skill": "Machine Learning",
          "llm_role": null,
          "roles_from_db": [
            {
              "display_name": "AI Engineer",
              "id": 13,
              "rationale": null,
              "role_archetype": null,
              "slug": "ai-engineer",
              "source": "db"
            },
            {
              "display_name": "ML Engineer",
              "id": 3,
              "rationale": null,
              "role_archetype": null,
              "slug": "ml-engineer",
              "source": "db"
            },
            {
              "display_name": "MLOps Engineer",
              "id": 16,
              "rationale": null,
              "role_archetype": null,
              "slug": "ml-ops-engineer",
              "source": "db"
            }
          ]
        },
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "React Frontend Development",
            "id": 96,
            "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
            "slug": "d_init_01",
            "source": "db"
          },
          "input_skill": "Machine Learning",
          "llm_role": null,
          "roles_from_db": []
        }
      ],
      "input_skill": "Machine Learning",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [],
      "canonical": null,
      "dimensions": [],
      "input_skill": "SOA",
      "matched_via": null,
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": {
        "derived": {
          "category": "Architecture Concepts",
          "skill_nature": "CONCEPT",
          "sub_category": "general",
          "typical_lifespan": "MULTI_YEAR",
          "version_strategy": "UNVERSIONED",
          "volatility": "MEDIUM"
        },
        "enrichment": null,
        "keep_log": [],
        "locked_dimensions": [],
        "merge_log": [],
        "placed": null,
        "relationships": null,
        "skill_id": "soa",
        "split_log": [],
        "typed": null,
        "warnings": []
      },
      "source_tag": "llm",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [],
      "canonical": null,
      "dimensions": [],
      "input_skill": "API Development",
      "matched_via": null,
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": {
        "derived": {
          "category": "API Tools",
          "skill_nature": "PRACTICE",
          "sub_category": "general",
          "typical_lifespan": "MULTI_YEAR",
          "version_strategy": "UNVERSIONED",
          "volatility": "MEDIUM"
        },
        "enrichment": null,
        "keep_log": [],
        "locked_dimensions": [],
        "merge_log": [],
        "placed": null,
        "relationships": null,
        "skill_id": "api-development",
        "split_log": [],
        "typed": null,
        "warnings": []
      },
      "source_tag": "llm",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "Agile",
          "alias_type": "CANONICAL",
          "id": 868,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 8,
        "display_name": "Agile",
        "id": 520,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "METHODOLOGY",
        "slug": "agile",
        "sub_category_id": 367,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "React Frontend Development",
            "id": 96,
            "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
            "slug": "d_init_01",
            "source": "db"
          },
          "input_skill": "Agile",
          "llm_role": null,
          "roles_from_db": []
        }
      ],
      "input_skill": "Agile",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [
        {
          "alias_text": "Scrum",
          "alias_type": "CANONICAL",
          "id": 1851,
          "is_primary": false,
          "match_strategy": "CASE_INSENSITIVE"
        }
      ],
      "canonical": {
        "category_id": 8,
        "display_name": "Scrum",
        "id": 1215,
        "is_also_category": false,
        "is_extractable": true,
        "skill_nature": "METHODOLOGY",
        "slug": "scrum",
        "sub_category_id": 921,
        "typical_lifespan": "EVERGREEN",
        "volatility": "STABLE"
      },
      "dimensions": [
        {
          "dimension": {
            "difficulty_hint": "well_known",
            "display_name": "React Frontend Development",
            "id": 96,
            "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
            "slug": "d_init_01",
            "source": "db"
          },
          "input_skill": "Scrum",
          "llm_role": null,
          "roles_from_db": []
        }
      ],
      "input_skill": "Scrum",
      "matched_via": "alias",
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": null,
      "source_tag": "db",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [],
      "canonical": null,
      "dimensions": [],
      "input_skill": "MLlib",
      "matched_via": null,
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": {
        "derived": {
          "category": "Machine Learning Frameworks",
          "skill_nature": "TOOL",
          "sub_category": "general",
          "typical_lifespan": "MULTI_YEAR",
          "version_strategy": "UNVERSIONED",
          "volatility": "MEDIUM"
        },
        "enrichment": null,
        "keep_log": [],
        "locked_dimensions": [],
        "merge_log": [],
        "placed": null,
        "relationships": null,
        "skill_id": "mllib",
        "split_log": [],
        "typed": null,
        "warnings": []
      },
      "source_tag": "llm",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [],
      "canonical": null,
      "dimensions": [],
      "input_skill": "GraphX",
      "matched_via": null,
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": {
        "derived": {
          "category": "Machine Learning Frameworks",
          "skill_nature": "TOOL",
          "sub_category": "general",
          "typical_lifespan": "MULTI_YEAR",
          "version_strategy": "UNVERSIONED",
          "volatility": "MEDIUM"
        },
        "enrichment": null,
        "keep_log": [],
        "locked_dimensions": [],
        "merge_log": [],
        "placed": null,
        "relationships": null,
        "skill_id": "graphx",
        "split_log": [],
        "typed": null,
        "warnings": []
      },
      "source_tag": "llm",
      "was_in_llm_skills": true
    },
    {
      "aliases_in_db": [],
      "canonical": null,
      "dimensions": [],
      "input_skill": "Pandas",
      "matched_via": null,
      "new_alias_persisted": false,
      "new_alias_text": null,
      "new_skill_meta": {
        "derived": {
          "category": "Data Analysis Libraries",
          "skill_nature": "TOOL",
          "sub_category": "general",
          "typical_lifespan": "EVERGREEN",
          "version_strategy": "UNVERSIONED",
          "volatility": "STABLE"
        },
        "enrichment": null,
        "keep_log": [],
        "locked_dimensions": [],
        "merge_log": [],
        "placed": null,
        "relationships": null,
        "skill_id": "pandas",
        "split_log": [],
        "typed": null,
        "warnings": []
      },
      "source_tag": "llm",
      "was_in_llm_skills": true
    }
  ],
  "unmatched_skills": [
    "Data Engineering",
    "Data Architecture",
    "EMR",
    "Unix",
    "Shell",
    "Oozie",
    "Apache Storm",
    "NiFi",
    "D3",
    "Master Data Management",
    "SOA",
    "API Development",
    "MLlib",
    "GraphX",
    "Pandas"
  ]
}
API 3 — final-role-output
{
  "chosen_role": {
    "display_name": "Data Engineer",
    "id": 2,
    "rationale": "Exact alias hit on data-engineer (1.0) \u2014 no other alias at this confidence; skill_top data-engineer 0.42 does not contradict",
    "role_archetype": null,
    "slug": "data-engineer",
    "source": "db"
  },
  "chosen_role_resolution": "in_db",
  "final_input_skills": [
    {
      "skill": "Data Engineering",
      "tag": "new"
    },
    {
      "skill": "Data Architecture",
      "tag": "new"
    },
    {
      "skill": "SQL",
      "tag": "in_db"
    },
    {
      "skill": "Python",
      "tag": "in_db"
    },
    {
      "skill": "Apache Spark",
      "tag": "in_db"
    },
    {
      "skill": "Hive",
      "tag": "in_db"
    },
    {
      "skill": "Scala",
      "tag": "in_db"
    },
    {
      "skill": "Hadoop",
      "tag": "in_db"
    },
    {
      "skill": "EMR",
      "tag": "new"
    },
    {
      "skill": "NoSQL",
      "tag": "in_db"
    },
    {
      "skill": "Unix",
      "tag": "new"
    },
    {
      "skill": "Shell",
      "tag": "new"
    },
    {
      "skill": "Oozie",
      "tag": "new"
    },
    {
      "skill": "Airflow",
      "tag": "in_db"
    },
    {
      "skill": "Apache Storm",
      "tag": "new"
    },
    {
      "skill": "Spark Streaming",
      "tag": "in_db"
    },
    {
      "skill": "NiFi",
      "tag": "new"
    },
    {
      "skill": "Tableau",
      "tag": "in_db"
    },
    {
      "skill": "Power BI",
      "tag": "in_db"
    },
    {
      "skill": "D3",
      "tag": "new"
    },
    {
      "skill": "Git",
      "tag": "in_db"
    },
    {
      "skill": "CI/CD",
      "tag": "in_db"
    },
    {
      "skill": "Master Data Management",
      "tag": "new"
    },
    {
      "skill": "Machine Learning",
      "tag": "in_db"
    },
    {
      "skill": "SOA",
      "tag": "new"
    },
    {
      "skill": "API Development",
      "tag": "new"
    },
    {
      "skill": "Agile",
      "tag": "in_db"
    },
    {
      "skill": "Scrum",
      "tag": "in_db"
    },
    {
      "skill": "MLlib",
      "tag": "new"
    },
    {
      "skill": "GraphX",
      "tag": "new"
    },
    {
      "skill": "Pandas",
      "tag": "new"
    }
  ],
  "llm_cost_api1_usd": null,
  "llm_cost_api2_usd": null,
  "llm_cost_api3_usd": null,
  "llm_cost_total_usd": null,
  "persistence": {
    "items": [
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Pega Programming Languages \u0026 DSLs",
          "id": 267,
          "rationale": "Programming languages and domain-specific languages used in Pega development.",
          "slug": "pega-programming-languages-dsls",
          "source": "db"
        },
        "dimension_id": 267,
        "input_skill": "SQL",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "Pega Developer",
            "id": 24,
            "rationale": null,
            "role_archetype": null,
            "slug": "pega-developer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 101,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Programming Languages for Data Work",
          "id": 21,
          "rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
          "slug": "programming-languages-for-data-work",
          "source": "db"
        },
        "dimension_id": 21,
        "input_skill": "SQL",
        "llm_role": null,
        "matched_chosen_role": true,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
        "role_dimension_saved": true,
        "roles_from_db": [
          {
            "display_name": "Data Engineer",
            "id": 2,
            "rationale": null,
            "role_archetype": null,
            "slug": "data-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 101,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Cloud Security Scripting \u0026 DSL Languages",
          "id": 248,
          "rationale": "Proficiency in programming and domain-specific languages used to automate and script cloud security controls.",
          "slug": "cloud-security-scripting-dsl-languages",
          "source": "db"
        },
        "dimension_id": 248,
        "input_skill": "Python",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "Cloud Security Engineer",
            "id": 23,
            "rationale": null,
            "role_archetype": null,
            "slug": "cloud-security-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 5,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Programming Languages",
          "id": 1,
          "rationale": "Primary implementation languages used to build client and server feature code. Full stack engineers need enough fluency to move across layers and implement product behavior end to end.",
          "slug": "programming-languages",
          "source": "db"
        },
        "dimension_id": 1,
        "input_skill": "Python",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "Backend Developer",
            "id": 1,
            "rationale": null,
            "role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
            "slug": "backend-engineer",
            "source": "db"
          },
          {
            "display_name": "Fullstack Developer",
            "id": 435,
            "rationale": null,
            "role_archetype": "Engineering",
            "slug": "fullstack-developer",
            "source": "db"
          },
          {
            "display_name": "Fullstack Developer",
            "id": 15,
            "rationale": null,
            "role_archetype": null,
            "slug": "full-stack-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 5,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Programming Languages and Scripting",
          "id": 59,
          "rationale": "Languages used to write security automation, analysis scripts, detection logic, and remediation helpers. This is the primary implementation surface for a cybersecurity engineer across tooling and response workflows.",
          "slug": "programming-languages-and-scripting",
          "source": "db"
        },
        "dimension_id": 59,
        "input_skill": "Python",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "Cyber Security Engineer",
            "id": 5,
            "rationale": null,
            "role_archetype": null,
            "slug": "cybersecurity-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 5,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Programming Languages for Data Work",
          "id": 21,
          "rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
          "slug": "programming-languages-for-data-work",
          "source": "db"
        },
        "dimension_id": 21,
        "input_skill": "Python",
        "llm_role": null,
        "matched_chosen_role": true,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
        "role_dimension_saved": true,
        "roles_from_db": [
          {
            "display_name": "Data Engineer",
            "id": 2,
            "rationale": null,
            "role_archetype": null,
            "slug": "data-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 5,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Programming Languages for ML Systems",
          "id": 39,
          "rationale": "Languages used to build training code, inference services, evaluation jobs, and ML glue code. This is the primary implementation surface for ML engineers across experimentation and productionization.",
          "slug": "programming-languages-for-ml-systems",
          "source": "db"
        },
        "dimension_id": 39,
        "input_skill": "Python",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "ML Engineer",
            "id": 3,
            "rationale": null,
            "role_archetype": null,
            "slug": "ml-engineer",
            "source": "db"
          },
          {
            "display_name": "MLOps Engineer",
            "id": 16,
            "rationale": null,
            "role_archetype": null,
            "slug": "ml-ops-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 5,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Programming Languages for XR",
          "id": 97,
          "rationale": "Primary implementation languages used to build immersive client features, interaction logic, and device-specific runtime behavior. This is the core coding surface for AR/VR experiences.",
          "slug": "programming-languages-for-xr",
          "source": "db"
        },
        "dimension_id": 97,
        "input_skill": "Python",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "AR/VR Engineer",
            "id": 8,
            "rationale": null,
            "role_archetype": null,
            "slug": "ar-vr-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 5,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Python Programming",
          "id": 290,
          "rationale": "Core Python language skills used to implement backend business logic, request handlers, integrations, and service internals. This is the primary coding surface for the role.",
          "slug": "python-programming",
          "source": "db"
        },
        "dimension_id": 290,
        "input_skill": "Python",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "Python Backend Developer",
            "id": 80,
            "rationale": null,
            "role_archetype": "Engineering",
            "slug": "python-backend-developer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 5,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "ETL and ELT Tooling",
          "id": 24,
          "rationale": "Packaged tools for extracting, loading, and transforming data across systems. This dimension covers connector-based ingestion, transformation frameworks, and managed integration products.",
          "slug": "etl-and-elt-tooling",
          "source": "db"
        },
        "dimension_id": 24,
        "input_skill": "Apache Spark",
        "llm_role": null,
        "matched_chosen_role": true,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
        "role_dimension_saved": true,
        "roles_from_db": [
          {
            "display_name": "Data Engineer",
            "id": 2,
            "rationale": null,
            "role_archetype": null,
            "slug": "data-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 1350,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Local Persistence and Offline Behavior",
          "id": 85,
          "rationale": "On-device storage used for caching, offline support, and durable client state. This cluster is coherent because iOS apps often need to preserve user progress and data when connectivity is limited.",
          "slug": "local-persistence-and-offline-behavior",
          "source": "db"
        },
        "dimension_id": 85,
        "input_skill": "Hive",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "Android Developer",
            "id": 4,
            "rationale": null,
            "role_archetype": null,
            "slug": "android-engineer",
            "source": "db"
          },
          {
            "display_name": "Flutter Developer",
            "id": 74,
            "rationale": null,
            "role_archetype": "Engineering",
            "slug": "flutter-developer",
            "source": "db"
          },
          {
            "display_name": "Hybrid Mobile Developer",
            "id": 11,
            "rationale": null,
            "role_archetype": null,
            "slug": "hybrid-mobile-developer",
            "source": "db"
          },
          {
            "display_name": "Native Mobile Developer",
            "id": 75,
            "rationale": null,
            "role_archetype": "Engineering",
            "slug": "native-mobile-developer",
            "source": "db"
          },
          {
            "display_name": "React Native Developer",
            "id": 73,
            "rationale": null,
            "role_archetype": "Engineering",
            "slug": "react-native-developer",
            "source": "db"
          },
          {
            "display_name": "iOS Developer",
            "id": 6,
            "rationale": null,
            "role_archetype": null,
            "slug": "ios-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 2754,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Programming Languages for Data Work",
          "id": 21,
          "rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
          "slug": "programming-languages-for-data-work",
          "source": "db"
        },
        "dimension_id": 21,
        "input_skill": "Scala",
        "llm_role": null,
        "matched_chosen_role": true,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
        "role_dimension_saved": true,
        "roles_from_db": [
          {
            "display_name": "Data Engineer",
            "id": 2,
            "rationale": null,
            "role_archetype": null,
            "slug": "data-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 102,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Programming Languages for ML Systems",
          "id": 39,
          "rationale": "Languages used to build training code, inference services, evaluation jobs, and ML glue code. This is the primary implementation surface for ML engineers across experimentation and productionization.",
          "slug": "programming-languages-for-ml-systems",
          "source": "db"
        },
        "dimension_id": 39,
        "input_skill": "Scala",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "ML Engineer",
            "id": 3,
            "rationale": null,
            "role_archetype": null,
            "slug": "ml-engineer",
            "source": "db"
          },
          {
            "display_name": "MLOps Engineer",
            "id": 16,
            "rationale": null,
            "role_archetype": null,
            "slug": "ml-ops-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 102,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "ETL and ELT Tooling",
          "id": 24,
          "rationale": "Packaged tools for extracting, loading, and transforming data across systems. This dimension covers connector-based ingestion, transformation frameworks, and managed integration products.",
          "slug": "etl-and-elt-tooling",
          "source": "db"
        },
        "dimension_id": 24,
        "input_skill": "Hadoop",
        "llm_role": null,
        "matched_chosen_role": true,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
        "role_dimension_saved": true,
        "roles_from_db": [
          {
            "display_name": "Data Engineer",
            "id": 2,
            "rationale": null,
            "role_archetype": null,
            "slug": "data-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 1351,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "NoSQL Databases",
          "id": 19,
          "rationale": "Models and manages data using non-relational database systems.",
          "slug": "nosql-databases",
          "source": "db"
        },
        "dimension_id": 19,
        "input_skill": "NoSQL",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "Backend Developer",
            "id": 1,
            "rationale": null,
            "role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
            "slug": "backend-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 1346,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Workflow Orchestration for ML Pipelines",
          "id": 54,
          "rationale": "Workflow engines used to coordinate training, evaluation, deployment, and retraining jobs. This cluster covers dependencies, retries, scheduling, and pipeline composition for ML lifecycle automation.",
          "slug": "workflow-orchestration-for-ml-pipelines",
          "source": "db"
        },
        "dimension_id": 54,
        "input_skill": "Airflow",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "ML Engineer",
            "id": 3,
            "rationale": null,
            "role_archetype": null,
            "slug": "ml-engineer",
            "source": "db"
          },
          {
            "display_name": "MLOps Engineer",
            "id": 16,
            "rationale": null,
            "role_archetype": null,
            "slug": "ml-ops-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 265,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "Stream Processing Systems",
          "id": 25,
          "rationale": "Technologies for processing event streams and near-real-time data flows. This includes stream transformations, windowing, stateful processing, and stream-to-warehouse delivery patterns.",
          "slug": "stream-processing-systems",
          "source": "db"
        },
        "dimension_id": 25,
        "input_skill": "Spark Streaming",
        "llm_role": null,
        "matched_chosen_role": true,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
        "role_dimension_saved": true,
        "roles_from_db": [
          {
            "display_name": "Data Engineer",
            "id": 2,
            "rationale": null,
            "role_archetype": null,
            "slug": "data-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 121,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "BI and Visualization Tools",
          "id": 31,
          "rationale": "Tools used to expose curated data to analysts and business users through dashboards, reports, and semantic exploration. Data engineers support these tools by shaping reliable datasets and performant models.",
          "slug": "bi-and-visualization-tools",
          "source": "db"
        },
        "dimension_id": 31,
        "input_skill": "Tableau",
        "llm_role": null,
        "matched_chosen_role": true,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
        "role_dimension_saved": true,
        "roles_from_db": [
          {
            "display_name": "Data Engineer",
            "id": 2,
            "rationale": null,
            "role_archetype": null,
            "slug": "data-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 150,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "BI and Visualization Tools",
          "id": 31,
          "rationale": "Tools used to expose curated data to analysts and business users through dashboards, reports, and semantic exploration. Data engineers support these tools by shaping reliable datasets and performant models.",
          "slug": "bi-and-visualization-tools",
          "source": "db"
        },
        "dimension_id": 31,
        "input_skill": "Power BI",
        "llm_role": null,
        "matched_chosen_role": true,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
        "role_dimension_saved": true,
        "roles_from_db": [
          {
            "display_name": "Data Engineer",
            "id": 2,
            "rationale": null,
            "role_archetype": null,
            "slug": "data-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 151,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "React Frontend Development",
          "id": 96,
          "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
          "slug": "d_init_01",
          "source": "db"
        },
        "dimension_id": 96,
        "input_skill": "Git",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [],
        "skill_dimension_saved": true,
        "skill_id": 1002,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "CI/CD Pipeline Platforms",
          "id": 150,
          "rationale": "Systems used to define, run, and maintain automated build and deployment workflows. This cluster is coherent because the role owns delivery automation end to end, including pipeline reliability and promotion logic.",
          "slug": "ci-cd-pipeline-platforms",
          "source": "db"
        },
        "dimension_id": 150,
        "input_skill": "CI/CD",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "DevOps Engineer",
            "id": 10,
            "rationale": null,
            "role_archetype": null,
            "slug": "devops-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 1190,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "CI/CD for Machine Learning",
          "id": 56,
          "rationale": "Tools and platforms for automating ML model integration, testing, and deployment pipelines.",
          "slug": "ci-cd-for-machine-learning",
          "source": "db"
        },
        "dimension_id": 56,
        "input_skill": "CI/CD",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "ML Engineer",
            "id": 3,
            "rationale": null,
            "role_archetype": null,
            "slug": "ml-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 1190,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "AI Governance and Model Security",
          "id": 50,
          "rationale": "Controls and documentation used to make models safer, auditable, and compliant. ML engineers use this to manage model risk, supply chain integrity, and governance requirements.",
          "slug": "ai-governance-and-model-security",
          "source": "db"
        },
        "dimension_id": 50,
        "input_skill": "Machine Learning",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [
          {
            "display_name": "AI Engineer",
            "id": 13,
            "rationale": null,
            "role_archetype": null,
            "slug": "ai-engineer",
            "source": "db"
          },
          {
            "display_name": "ML Engineer",
            "id": 3,
            "rationale": null,
            "role_archetype": null,
            "slug": "ml-engineer",
            "source": "db"
          },
          {
            "display_name": "MLOps Engineer",
            "id": 16,
            "rationale": null,
            "role_archetype": null,
            "slug": "ml-ops-engineer",
            "source": "db"
          }
        ],
        "skill_dimension_saved": true,
        "skill_id": 1356,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "React Frontend Development",
          "id": 96,
          "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
          "slug": "d_init_01",
          "source": "db"
        },
        "dimension_id": 96,
        "input_skill": "Machine Learning",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [],
        "skill_dimension_saved": true,
        "skill_id": 1356,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "React Frontend Development",
          "id": 96,
          "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
          "slug": "d_init_01",
          "source": "db"
        },
        "dimension_id": 96,
        "input_skill": "Agile",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [],
        "skill_dimension_saved": true,
        "skill_id": 520,
        "skill_tag": "in_db",
        "skipped_reason": null
      },
      {
        "chosen_role_id": 2,
        "dimension": {
          "difficulty_hint": "well_known",
          "display_name": "React Frontend Development",
          "id": 96,
          "rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
          "slug": "d_init_01",
          "source": "db"
        },
        "dimension_id": 96,
        "input_skill": "Scrum",
        "llm_role": null,
        "matched_chosen_role": false,
        "outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
        "role_dimension_saved": false,
        "roles_from_db": [],
        "skill_dimension_saved": true,
        "skill_id": 1215,
        "skill_tag": "in_db",
        "skipped_reason": null
      }
    ],
    "new_skills_created": 0,
    "role_dimension_saved": 0,
    "skill_dimension_saved": 0,
    "skipped": 0
  },
  "planner_output": null,
  "run_id": "459d8d5b-8e49-4c9b-a801-9389a9c4c4ae"
}

LLM Calls

Every model call made for this run, in pipeline order. Click a card to see the model's response.

Loading…