Pipeline run
519de9f6-c0e1-4fae-8508-3a9dfd6a185c
Client output enrichment
v2 Skill cluster · Nature of work · AI index · Tech stack maturity · Evidence · KRA descriptionvocab breakdown (legacy)
Signals
Post-classification
Captured for admin review
1 POST /skills/extract-from-jd
2 POST /skills/extract-details
3 POST /skills/final-role-output
Data Engineer
CASE Aslug: data-engineer · id: 2 · source: db
Exact alias hit on data-engineer (1.0) — no other alias at this confidence; skill_top data-engineer 0.33 does not contradict
Resolution:
in_db
— role exists in library; skill↔dim and role↔dim links saved when applicable.
Job description
We are looking for a highly skilled and motivated Data Engineer to join our team. The Data Engineer will play a crucial role in the design, development, and maintenance of our data infrastructure, with a focus on ETL/Ingestion processes and system health. Key Responsibilities: ETL/Ingestion: Design, implement, and optimize data extraction, transformation, and loading (ETL) processes to ingest data from various sources into our data warehouse. Data Engineer: Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and build data pipelines that enable efficient data analysis. Maintenance, Upgrade, and System Health Check: Conduct routine system health checks, perform system upgrades, and address any maintenance requirements to ensure data pipeline stability and performance. Services/Resources Utilized: Work with the following services and resources, including but not limited to: BigQuery Cloud Data Fusion Composer and Airflow Cloud Run Google Cloud Storage Git and BitBucket Cloud Build Mandatory Skills and Qualifications: Strong proficiency in using BigQuery for data processing and analysis. Experience with ETL tools and techniques, particularly with Cloud Data Fusion. Proficiency in managing and scheduling workflows using Composer and Airflow. Deployment and orchestration experience with Cloud Run. Broader areas of services ETL / Ingestion, Data Engineer Maintenance, upgrade, system health check up Services / Resources utilised: BigQuery Cloud Data Fusion Composer, Airflow Cloud Run Google Cloud Storage Production Deployment: Git BitBucket Cloud Build Services / Resources utilised (Mandatory): BigQuery ,ETL Cloud Data Fusion Composer, Airflow Cloud Run Google Cloud Storage SQL, Python Education and Experience: Bachelor's degree in Computer Science, Information Systems, or a related field (Master's degree preferred). 2-3 years of experience as a Data Engineer or in a similar role, with a proven track record of designing and implementing ETL processes.
Skills from this JD
Each row merges API 1 extraction, API 2 library match / v3 orchestration (dimensions + locked dims), and API 3 persistence tags.
Aliases — catalog
- BigQuery (CANONICAL) primary
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Service
- Sub-category
- Data Warehouse Service
- Vendor
- License
- proprietary
- Year introduced
- 2011
- Confidence
- 0.98
- Version strategy
- NOT_APPLICABLE
Maturity reasoning: BigQuery appears frequently in data/analytics job descriptions and is a core Google Cloud warehouse offering, with broad enterprise adoption and strong ecosystem support.
Skill profile (library / DB)
- Skill nature
- CLOUD_SERVICE
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 11
- Sub-category id
- 118
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
Cloud Data Warehouses Catalog dimension db id 22
Library dimension (catalog)
Roles linked in library: Data Engineer
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
Cloud Data Warehouses
cloud-data-warehouses
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved |
Skill enrichment (orchestrator / LLM)
No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).
- Category
- Cloud Platforms
- Sub-category
- general
- Skill nature
- PLATFORM
- Volatility
- MEDIUM
- Typical lifespan
- MULTI_YEAR
- Version strategy
- UNVERSIONED
Aliases — catalog
- Composer (CANONICAL) primary
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Tool
- Sub-category
- Dependency Manager Tool
- Vendor
- Nils Adermann
- License
- mit
- Year introduced
- 2011
- Confidence
- 0.97
- Version strategy
- NOT_APPLICABLE
Maturity reasoning: Composer is the standard PHP dependency manager and appears routinely in PHP job descriptions and framework docs; Packagist ecosystem usage remains broad, with no successor replacing it.
Skill profile (library / DB)
- Skill nature
- TOOL
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 13
- Sub-category id
- 2456
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
Composer and Dependency Management Catalog dimension db id 355
Library dimension (catalog)
Roles linked in library: Drupal Dev
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
Composer and Dependency Management
composer-and-dependency-management
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
Aliases — catalog
- Airflow (CANONICAL) primary
- airflow 2 (VERSION)
- airflow-2 (VERSION)
- airflow2 (VERSION)
- airflow2.x (VERSION)
- apache airflow 2 (VERSION)
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Tool
- Sub-category
- Workflow Orchestration Tool
- Vendor
- Apache Software Foundation
- License
- apache_2
- Year introduced
- 2014
- Confidence
- 0.95
- Version strategy
- SEPARATE_ENTITY
- Version tag
- 2.x
Maturity reasoning: Apache Airflow appears in many data engineering job postings and is a common orchestration choice in production stacks; its GitHub activity and ecosystem remain strong, with no vendor sunset or clear replacement dominating JDs.
Skill profile (library / DB)
- Skill nature
- TOOL
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 13
- Sub-category id
- 130
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
Workflow Orchestration for ML Pipelines Catalog dimension db id 54
Library dimension (catalog)
Roles linked in library: ML Engineer, MLOps Engineer
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
Workflow Orchestration for ML Pipelines
workflow-orchestration-for-ml-pipelines
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
Skill enrichment (orchestrator / LLM)
No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).
- Category
- Cloud Platforms
- Sub-category
- general
- Skill nature
- PLATFORM
- Volatility
- MEDIUM
- Typical lifespan
- MULTI_YEAR
- Version strategy
- UNVERSIONED
Aliases — catalog
- Google Cloud Storage (CANONICAL) primary
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Service
- Sub-category
- Object Storage Service
- Vendor
- License
- proprietary
- Year introduced
- 2010
- Confidence
- 0.98
- Version strategy
- NOT_APPLICABLE
Maturity reasoning: Broadly used object storage on GCP; appears frequently in cloud/data engineering JDs and is a standard managed service alongside S3/Azure Blob.
Skill profile (library / DB)
- Skill nature
- CLOUD_SERVICE
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 11
- Sub-category id
- 120
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
Cloud Storage and Data Services Catalog dimension db id 144
Library dimension (catalog)
Roles linked in library: Cloud Architect
-
Cloud Storage and File Formats Catalog dimension db id 35
Library dimension (catalog)
Roles linked in library: Data Engineer
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
Cloud Storage and Data Services
cloud-storage-and-data-services
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Cloud Storage and File Formats
cloud-storage-and-file-formats
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved |
Aliases — catalog
- Git (CANONICAL)
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Tool
- Sub-category
- Version Control Tool
- Vendor
- Linus Torvalds
- License
- gpl_v2
- Year introduced
- 2005
- Confidence
- 0.99
- Version strategy
- NOT_APPLICABLE
Maturity reasoning: Git is a hiring-pipeline staple: it appears in the vast majority of software engineering job descriptions and is the default VCS on GitHub/GitLab/Bitbucket.
Skill profile (library / DB)
- Skill nature
- TOOL
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 13
- Sub-category id
- 730
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
React Frontend Development Catalog dimension db id 96
Library dimension (catalog)
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
React Frontend Development
d_init_01
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
Skill enrichment (orchestrator / LLM)
No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).
- Category
- DevOps Tools
- Sub-category
- general
- Skill nature
- TOOL
- Volatility
- MEDIUM
- Typical lifespan
- MULTI_YEAR
- Version strategy
- UNVERSIONED
Skill enrichment (orchestrator / LLM)
No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).
- Category
- Cloud Platforms
- Sub-category
- general
- Skill nature
- PLATFORM
- Volatility
- MEDIUM
- Typical lifespan
- MULTI_YEAR
- Version strategy
- UNVERSIONED
Aliases — catalog
- SQL (CANONICAL) primary
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Language
- Sub-category
- Query Language
- Vendor
- ANSI
- License
- unknown
- Year introduced
- 1974
- Confidence
- 0.99
- Version strategy
- NOT_APPLICABLE
Maturity reasoning: SQL appears in a large share of data, backend, and analytics job descriptions and remains the default query language for PostgreSQL, MySQL, and cloud warehouses like Snowflake/BigQuery.
Skill profile (library / DB)
- Skill nature
- LANGUAGE
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 6
- Sub-category id
- 97
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
Pega Programming Languages & DSLs Catalog dimension db id 267
Library dimension (catalog)
Roles linked in library: Pega Developer
-
Programming Languages & DSLs Catalog dimension db id 475
Library dimension (catalog)
Roles linked in library: Engineering Manager
-
Programming Languages for Data Work Catalog dimension db id 21
Library dimension (catalog)
Roles linked in library: Data Engineer
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
Pega Programming Languages & DSLs
pega-programming-languages-dsls
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Programming Languages & DSLs
programming-languages-dsls
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Programming Languages for Data Work
programming-languages-for-data-work
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved |
Aliases — catalog
- Python (CANONICAL) primary
- Python 2 (VERSION)
- Python 2.x (VERSION)
- Python 3 (VERSION)
- Python 3.10 (VERSION)
- Python 3.11 (VERSION)
- Python 3.12 (VERSION)
- Python 3.x (VERSION)
- py (VERSION)
- py2 (VERSION)
- py3 (VERSION)
- python 3 (VERSION)
- python 3.x (VERSION)
- python2 (VERSION)
- python3 (VERSION)
- python3.x (VERSION)
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Language
- Sub-category
- Programming Language
- Vendor
- PSF
- License
- mit
- Year introduced
- 1991
- Confidence
- 0.99
- Version strategy
- SEPARATE_ENTITY
- Version tag
- 3
Maturity reasoning: Python appears in a very high volume of job descriptions across data, backend, automation, and ML roles, and remains a default hiring-pipeline language on major job boards and tech stacks.
Skill profile (library / DB)
- Skill nature
- LANGUAGE
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 6
- Sub-category id
- 96
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
Cloud Security Scripting & DSL Languages Catalog dimension db id 248
Library dimension (catalog)
Roles linked in library: Cloud Security Engineer
-
Programming Languages Catalog dimension db id 1
Library dimension (catalog)
Roles linked in library: Backend Developer, Fullstack Developer, Fullstack Developer
-
Programming Languages & DSLs Catalog dimension db id 475
Library dimension (catalog)
Roles linked in library: Engineering Manager
-
Programming Languages and Scripting Catalog dimension db id 59
Library dimension (catalog)
Roles linked in library: Cyber Security Engineer
-
Programming Languages for Data Work Catalog dimension db id 21
Library dimension (catalog)
Roles linked in library: Data Engineer
-
Programming Languages for ML Systems Catalog dimension db id 39
Library dimension (catalog)
Roles linked in library: ML Engineer, MLOps Engineer
-
Programming Languages for XR Catalog dimension db id 97
Library dimension (catalog)
Roles linked in library: AR/VR Engineer
-
Python Programming Catalog dimension db id 290
Library dimension (catalog)
Roles linked in library: Python Backend Developer
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
Cloud Security Scripting & DSL Languages
cloud-security-scripting-dsl-languages
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Programming Languages
programming-languages
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Programming Languages & DSLs
programming-languages-dsls
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Programming Languages and Scripting
programming-languages-and-scripting
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Programming Languages for Data Work
programming-languages-for-data-work
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved |
|
Programming Languages for ML Systems
programming-languages-for-ml-systems
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Programming Languages for XR
programming-languages-for-xr
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Python Programming
python-programming
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
Skill enrichment (orchestrator / LLM)
No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).
- Category
- Data Engineering Tools
- Sub-category
- general
- Skill nature
- CONCEPT
- Volatility
- MEDIUM
- Typical lifespan
- MULTI_YEAR
- Version strategy
- UNVERSIONED
All API 3 persistence rows
Same grid as the skill-extractor “Persistence items” table: one row per (skill × dimension) work item.
| Skill | Tag | Dimension | Skill↔dim | Role↔dim | Outcome | Notes |
|---|---|---|---|---|---|---|
| BigQuery | in_db |
Cloud Data Warehouses
cloud-data-warehouses
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved | |
| Composer | in_db |
Composer and Dependency Management
composer-and-dependency-management
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| Airflow | in_db |
Workflow Orchestration for ML Pipelines
workflow-orchestration-for-ml-pipelines
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| Google Cloud Storage | in_db |
Cloud Storage and Data Services
cloud-storage-and-data-services
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| Google Cloud Storage | in_db |
Cloud Storage and File Formats
cloud-storage-and-file-formats
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved | |
| Git | in_db |
React Frontend Development
d_init_01
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| SQL | in_db |
Pega Programming Languages & DSLs
pega-programming-languages-dsls
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| SQL | in_db |
Programming Languages & DSLs
programming-languages-dsls
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| SQL | in_db |
Programming Languages for Data Work
programming-languages-for-data-work
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved | |
| Python | in_db |
Cloud Security Scripting & DSL Languages
cloud-security-scripting-dsl-languages
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| Python | in_db |
Programming Languages
programming-languages
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| Python | in_db |
Programming Languages & DSLs
programming-languages-dsls
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| Python | in_db |
Programming Languages and Scripting
programming-languages-and-scripting
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| Python | in_db |
Programming Languages for Data Work
programming-languages-for-data-work
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved | |
| Python | in_db |
Programming Languages for ML Systems
programming-languages-for-ml-systems
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| Python | in_db |
Programming Languages for XR
programming-languages-for-xr
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| Python | in_db |
Python Programming
python-programming
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
Library artifacts (this run)
| Kind | Detail | DB id |
|---|---|---|
| canonical_skill_proposed | Cloud Data Fusion | type=Cloud Platforms subtype=general nature=PLATFORM lifespan=MULTI_YEAR | |
| canonical_skill_proposed | Cloud Run | type=Cloud Platforms subtype=general nature=PLATFORM lifespan=MULTI_YEAR | |
| canonical_skill_proposed | Bitbucket | type=DevOps Tools subtype=general nature=TOOL lifespan=MULTI_YEAR | |
| canonical_skill_proposed | Cloud Build | type=Cloud Platforms subtype=general nature=PLATFORM lifespan=MULTI_YEAR | |
| canonical_skill_proposed | ETL | type=Data Engineering Tools subtype=general nature=CONCEPT lifespan=MULTI_YEAR |
nano JD Parser — gpt-4.1-nano click to toggle
Show raw JSON
{
"JD_type": "pass",
"about_company": null,
"certifications": [],
"company_name": null,
"ctc": null,
"domain": {
"primary": {
"aliases": [],
"domain": "IT Services \u0026 Consulting"
},
"secondary": null
},
"education": [
{
"level": "Bachelor\u0027s",
"qualification": "BTECH/BE - Computer Science (or related)",
"raw": "Bachelor\u0027s degree in Computer Science, Information Systems, or a related field (Master\u0027s degree preferred)",
"requirement": "required"
}
],
"experience": {
"max": 3,
"min": 2,
"raw": "2-3 years of experience as a Data Engineer or in a similar role"
},
"job_locations": [],
"role": "Data Engineer",
"role_aliases": [
"Data Engineer",
"ETL Engineer",
"Data Pipeline Engineer"
],
"role_archetype": "Data",
"roles_and_responsibilities": [
{
"bullet_count": 6,
"heading": "Key Responsibilities",
"heading_was_present": true,
"source_marker": {
"first_5_words": "ETL/Ingestion: Design, implement, and",
"last_5_words": "Cloud Data Fusion Composer and Airflow"
},
"text": "ETL/Ingestion: Design, implement, and optimize data extraction, transformation, and loading (ETL) processes to ingest data from various sources into our data warehouse.\n\nData Engineer: Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and build data pipelines that enable efficient data analysis.\n\nMaintenance, Upgrade, and System Health Check: Conduct routine system health checks, perform system upgrades, and address any maintenance requirements to ensure data pipeline stability and performance.\n\nServices/Resources Utilized: Work with the following services and resources, including but not limited to:\n\nBigQuery\nCloud Data Fusion\nComposer and Airflow\nCloud Run\nGoogle Cloud Storage\nGit and BitBucket\nCloud Build",
"word_count": 156
},
{
"bullet_count": 4,
"heading": "Mandatory Skills and Qualifications",
"heading_was_present": true,
"source_marker": {
"first_5_words": "Strong proficiency in using BigQuery",
"last_5_words": "and orchestration experience with Cloud Run."
},
"text": "Strong proficiency in using BigQuery for data processing and analysis.\nExperience with ETL tools and techniques, particularly with Cloud Data Fusion.\nProficiency in managing and scheduling workflows using Composer and Airflow.\nDeployment and orchestration experience with Cloud Run.",
"word_count": 48
},
{
"bullet_count": 0,
"heading": "Broader areas of services ETL / Ingestion, Data Engineer Maintenance, upgrade, system health check up",
"heading_was_present": true,
"source_marker": {
"first_5_words": "Services / Resources utilised: BigQuery",
"last_5_words": "Cloud Data Fusion Composer, Airflow Cloud"
},
"text": "Services / Resources utilised: BigQuery Cloud Data Fusion Composer, Airflow Cloud Run Google Cloud Storage Production Deployment: Git BitBucket Cloud Build\n\nServices / Resources utilised (Mandatory): BigQuery ,ETL Cloud Data Fusion Composer, Airflow Cloud Run Google Cloud Storage SQL, Python",
"word_count": 43
}
],
"urls": []
}
API 1 — extract-from-jd click to toggle
{
"final_skills": [
{
"is_primary": true,
"skill_name": "BigQuery"
},
{
"is_primary": true,
"skill_name": "Cloud Data Fusion"
},
{
"is_primary": true,
"skill_name": "Composer"
},
{
"is_primary": true,
"skill_name": "Airflow"
},
{
"is_primary": true,
"skill_name": "Cloud Run"
},
{
"is_primary": true,
"skill_name": "Google Cloud Storage"
},
{
"is_primary": true,
"skill_name": "Git"
},
{
"is_primary": true,
"skill_name": "Bitbucket"
},
{
"is_primary": true,
"skill_name": "Cloud Build"
},
{
"is_primary": true,
"skill_name": "SQL"
},
{
"is_primary": true,
"skill_name": "Python"
},
{
"is_primary": true,
"skill_name": "ETL"
}
],
"jd_role": {
"display_name": "Data Engineer",
"rationale": null,
"role_aliases": [
"Data Engineer",
"ETL Engineer",
"Data Pipeline Engineer"
],
"role_archetype": "Data",
"slug": ""
},
"nano_parsed": {
"JD_type": "pass",
"about_company": null,
"certifications": [],
"company_name": null,
"ctc": null,
"domain": {
"primary": {
"aliases": [],
"domain": "IT Services \u0026 Consulting"
},
"secondary": null
},
"education": [
{
"level": "Bachelor\u0027s",
"qualification": "BTECH/BE - Computer Science (or related)",
"raw": "Bachelor\u0027s degree in Computer Science, Information Systems, or a related field (Master\u0027s degree preferred)",
"requirement": "required"
}
],
"experience": {
"max": 3,
"min": 2,
"raw": "2-3 years of experience as a Data Engineer or in a similar role"
},
"job_locations": [],
"role": "Data Engineer",
"role_aliases": [
"Data Engineer",
"ETL Engineer",
"Data Pipeline Engineer"
],
"role_archetype": "Data",
"roles_and_responsibilities": [
{
"bullet_count": 6,
"heading": "Key Responsibilities",
"heading_was_present": true,
"source_marker": {
"first_5_words": "ETL/Ingestion: Design, implement, and",
"last_5_words": "Cloud Data Fusion Composer and Airflow"
},
"text": "ETL/Ingestion: Design, implement, and optimize data extraction, transformation, and loading (ETL) processes to ingest data from various sources into our data warehouse.\n\nData Engineer: Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and build data pipelines that enable efficient data analysis.\n\nMaintenance, Upgrade, and System Health Check: Conduct routine system health checks, perform system upgrades, and address any maintenance requirements to ensure data pipeline stability and performance.\n\nServices/Resources Utilized: Work with the following services and resources, including but not limited to:\n\nBigQuery\nCloud Data Fusion\nComposer and Airflow\nCloud Run\nGoogle Cloud Storage\nGit and BitBucket\nCloud Build",
"word_count": 156
},
{
"bullet_count": 4,
"heading": "Mandatory Skills and Qualifications",
"heading_was_present": true,
"source_marker": {
"first_5_words": "Strong proficiency in using BigQuery",
"last_5_words": "and orchestration experience with Cloud Run."
},
"text": "Strong proficiency in using BigQuery for data processing and analysis.\nExperience with ETL tools and techniques, particularly with Cloud Data Fusion.\nProficiency in managing and scheduling workflows using Composer and Airflow.\nDeployment and orchestration experience with Cloud Run.",
"word_count": 48
},
{
"bullet_count": 0,
"heading": "Broader areas of services ETL / Ingestion, Data Engineer Maintenance, upgrade, system health check up",
"heading_was_present": true,
"source_marker": {
"first_5_words": "Services / Resources utilised: BigQuery",
"last_5_words": "Cloud Data Fusion Composer, Airflow Cloud"
},
"text": "Services / Resources utilised: BigQuery Cloud Data Fusion Composer, Airflow Cloud Run Google Cloud Storage Production Deployment: Git BitBucket Cloud Build\n\nServices / Resources utilised (Mandatory): BigQuery ,ETL Cloud Data Fusion Composer, Airflow Cloud Run Google Cloud Storage SQL, Python",
"word_count": 43
}
],
"urls": []
},
"rejected": false,
"rejection_reason": null,
"run_id": "519de9f6-c0e1-4fae-8508-3a9dfd6a185c",
"stage3_signals": {
"alias_found": true,
"alias_match_roles": [
{
"display_name": "Data Engineer",
"kra_matches": null,
"matched_count": null,
"matched_skills": null,
"role_id": 2,
"score": 1.0,
"slug": "data-engineer",
"total_count": null
}
],
"kra_match_roles": [
{
"display_name": "Data Engineer",
"kra_matches": [
{
"kra_text": "Works with data analysts, data scientists, and business stakeholders to define data models, ingestion schedules, and data delivery requirements.",
"sentence": "Data Engineer: Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and build data pipelines that enable efficient data analysis.",
"similarity": 0.7124
},
{
"kra_text": "Works with data analysts, data scientists, and business stakeholders to define data models, ingestion schedules, and data delivery requirements.",
"sentence": "ETL/Ingestion: Design, implement, and optimize data extraction, transformation, and loading (ETL) processes to ingest data from various sources into our data warehouse.",
"similarity": 0.5864
},
{
"kra_text": "Monitors pipeline health, SLA breach alerts, and job failure notifications, and performs root cause analysis for data pipeline incidents.",
"sentence": "Maintenance, Upgrade, and System Health Check: Conduct routine system health checks, perform system upgrades, and address any maintenance requirements to ensure data pipeline stability and performance.",
"similarity": 0.5587
}
],
"matched_count": null,
"matched_skills": null,
"role_id": 2,
"score": 0.6191,
"slug": "data-engineer",
"total_count": null
},
{
"display_name": "MLOps Engineer",
"kra_matches": [
{
"kra_text": "Orchestrates model serving deployments to production using Kubernetes, MLflow Model Registry, SageMaker, or Kubeflow Serving infrastructure.",
"sentence": "Services / Resources utilised: BigQuery Cloud Data Fusion Composer, Airflow Cloud Run Google Cloud Storage Production Deployment: Git BitBucket Cloud Build",
"similarity": 0.518
},
{
"kra_text": "Orchestrates model serving deployments to production using Kubernetes, MLflow Model Registry, SageMaker, or Kubeflow Serving infrastructure.",
"sentence": "Deployment and orchestration experience with Cloud Run.",
"similarity": 0.5179
},
{
"kra_text": "Validates model performance benchmarks, data schema contracts, and system integration health before signing off on production release readiness.",
"sentence": "Maintenance, Upgrade, and System Health Check: Conduct routine system health checks, perform system upgrades, and address any maintenance requirements to ensure data pipeline stability and performance.",
"similarity": 0.5152
}
],
"matched_count": null,
"matched_skills": null,
"role_id": 16,
"score": 0.517,
"slug": "ml-ops-engineer",
"total_count": null
},
{
"display_name": "DevOps Engineer",
"kra_matches": [
{
"kra_text": "Manages container orchestration with Kubernetes and Docker, deploying applications as pods, managing namespaces, and configuring auto-scaling across cloud environments.",
"sentence": "Deployment and orchestration experience with Cloud Run.",
"similarity": 0.5339
},
{
"kra_text": "Monitors CI/CD pipeline reliability, identifies bottlenecks in delivery workflows, and improves deployment frequency, lead time, and failure recovery rate.",
"sentence": "Maintenance, Upgrade, and System Health Check: Conduct routine system health checks, perform system upgrades, and address any maintenance requirements to ensure data pipeline stability and performance.",
"similarity": 0.5156
},
{
"kra_text": "Builds and maintains CI/CD pipelines using Jenkins, GitHub Actions, GitLab CI, or CircleCI to automate build, test, security scanning, and deployment workflows.",
"sentence": "Services / Resources utilised: BigQuery Cloud Data Fusion Composer, Airflow Cloud Run Google Cloud Storage Production Deployment: Git BitBucket Cloud Build",
"similarity": 0.4813
}
],
"matched_count": null,
"matched_skills": null,
"role_id": 10,
"score": 0.5102,
"slug": "devops-engineer",
"total_count": null
},
{
"display_name": "ML Engineer",
"kra_matches": [
{
"kra_text": "Prepares, cleans, and transforms training datasets, manages feature stores, and builds feature engineering pipelines for model training.",
"sentence": "Data Engineer: Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and build data pipelines that enable efficient data analysis.",
"similarity": 0.5063
},
{
"kra_text": "Builds model serving infrastructure to deploy trained models as real-time prediction APIs or batch inference jobs using TorchServe, TensorFlow Serving, or SageMaker.",
"sentence": "Services / Resources utilised: BigQuery Cloud Data Fusion Composer, Airflow Cloud Run Google Cloud Storage Production Deployment: Git BitBucket Cloud Build",
"similarity": 0.4501
},
{
"kra_text": "Prepares, cleans, and transforms training datasets, manages feature stores, and builds feature engineering pipelines for model training.",
"sentence": "Services / Resources utilised (Mandatory): BigQuery ,ETL Cloud Data Fusion Composer, Airflow Cloud Run Google Cloud Storage SQL, Python",
"similarity": 0.4402
}
],
"matched_count": null,
"matched_skills": null,
"role_id": 3,
"score": 0.4655,
"slug": "ml-engineer",
"total_count": null
},
{
"display_name": "Cloud Architect",
"kra_matches": [
{
"kra_text": "Evaluates cloud-native managed services, serverless compute, PaaS databases, and CDN solutions for workload fit and total cost of ownership.",
"sentence": "Services / Resources utilised: BigQuery Cloud Data Fusion Composer, Airflow Cloud Run Google Cloud Storage Production Deployment: Git BitBucket Cloud Build",
"similarity": 0.4821
},
{
"kra_text": "Defines cloud adoption roadmaps, lift-and-shift vs. refactor migration strategies, and landing zone architectures for workloads moving to AWS, Azure, or GCP.",
"sentence": "Deployment and orchestration experience with Cloud Run.",
"similarity": 0.4664
},
{
"kra_text": "Evaluates cloud-native managed services, serverless compute, PaaS databases, and CDN solutions for workload fit and total cost of ownership.",
"sentence": "Services / Resources utilised (Mandatory): BigQuery ,ETL Cloud Data Fusion Composer, Airflow Cloud Run Google Cloud Storage SQL, Python",
"similarity": 0.4433
}
],
"matched_count": null,
"matched_skills": null,
"role_id": 9,
"score": 0.4639,
"slug": "cloud-architect",
"total_count": null
}
],
"skill_match_roles": [
{
"display_name": "Data Engineer",
"kra_matches": null,
"matched_count": 4,
"matched_skills": [
"BigQuery",
"Google Cloud Storage",
"Python",
"SQL"
],
"role_id": 2,
"score": 0.3333,
"slug": "data-engineer",
"total_count": 12
},
{
"display_name": "ML Engineer",
"kra_matches": null,
"matched_count": 2,
"matched_skills": [
"Airflow",
"Python"
],
"role_id": 3,
"score": 0.1667,
"slug": "ml-engineer",
"total_count": 12
},
{
"display_name": "Engineering Manager",
"kra_matches": null,
"matched_count": 2,
"matched_skills": [
"Python",
"SQL"
],
"role_id": 121,
"score": 0.1667,
"slug": "engineering-manager",
"total_count": 12
},
{
"display_name": "MLOps Engineer",
"kra_matches": null,
"matched_count": 2,
"matched_skills": [
"Airflow",
"Python"
],
"role_id": 16,
"score": 0.1667,
"slug": "ml-ops-engineer",
"total_count": 12
},
{
"display_name": "AR/VR Engineer",
"kra_matches": null,
"matched_count": 1,
"matched_skills": [
"Python"
],
"role_id": 8,
"score": 0.0833,
"slug": "ar-vr-engineer",
"total_count": 12
}
]
},
"stage4_decision": {
"alias_collision_detected": false,
"case": "A",
"chosen_role": {
"display_name": "Data Engineer",
"kra_matches": null,
"matched_count": null,
"matched_skills": null,
"role_id": 2,
"score": 1.0,
"slug": "data-engineer",
"total_count": null
},
"confidence": 1.0,
"is_new_role": false,
"llm2_fired": false,
"llm2_reasoning": null,
"matched_dimensions": [],
"matched_kras": [],
"matched_skills": [],
"new_role_display_name": null,
"new_role_slug": null,
"queued": false,
"reasoning": "Exact alias hit on data-engineer (1.0) \u2014 no other alias at this confidence; skill_top data-engineer 0.33 does not contradict",
"sub_role": null
},
"stage5_updates": {
"centroid_n_after": 323,
"centroid_updated": true,
"collision_log_id": null,
"new_kra_attached": null,
"new_skills_attached": [
{
"is_primary": true,
"queue_id": 15074,
"role_display_name": "Data Engineer",
"role_slug": "data-engineer",
"skill_name": "Cloud Data Fusion",
"status": "pending"
},
{
"is_primary": true,
"queue_id": 15075,
"role_display_name": "Data Engineer",
"role_slug": "data-engineer",
"skill_name": "Cloud Run",
"status": "pending"
},
{
"is_primary": true,
"queue_id": 15076,
"role_display_name": "Data Engineer",
"role_slug": "data-engineer",
"skill_name": "Bitbucket",
"status": "pending"
},
{
"is_primary": true,
"queue_id": 15077,
"role_display_name": "Data Engineer",
"role_slug": "data-engineer",
"skill_name": "Cloud Build",
"status": "pending"
},
{
"is_primary": true,
"queue_id": 15078,
"role_display_name": "Data Engineer",
"role_slug": "data-engineer",
"skill_name": "ETL",
"status": "pending"
}
],
"queue_entry_id": null,
"v3_pipeline_triggered": false,
"v3_role_slug": null,
"v3_run_id": null
}
}
API 2 — extract-details
{
"alias_matches": [
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 300,
"existing_alias_text": "BigQuery",
"input_term": "BigQuery",
"matched_canonical": {
"category_id": 11,
"display_name": "BigQuery",
"id": 106,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "CLOUD_SERVICE",
"slug": "bigquery",
"sub_category_id": 118,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "alias"
},
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 4603,
"existing_alias_text": "Composer",
"input_term": "Composer",
"matched_canonical": {
"category_id": 13,
"display_name": "Composer",
"id": 3126,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "TOOL",
"slug": "composer",
"sub_category_id": 2456,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "alias"
},
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 526,
"existing_alias_text": "Airflow",
"input_term": "Airflow",
"matched_canonical": {
"category_id": 13,
"display_name": "Airflow",
"id": 265,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "TOOL",
"slug": "airflow",
"sub_category_id": 130,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "alias"
},
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 380,
"existing_alias_text": "Google Cloud Storage",
"input_term": "Google Cloud Storage",
"matched_canonical": {
"category_id": 11,
"display_name": "Google Cloud Storage",
"id": 171,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "CLOUD_SERVICE",
"slug": "google-cloud-storage",
"sub_category_id": 120,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "alias"
},
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 1613,
"existing_alias_text": "Git",
"input_term": "Git",
"matched_canonical": {
"category_id": 13,
"display_name": "Git",
"id": 1002,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "TOOL",
"slug": "git",
"sub_category_id": 730,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "alias"
},
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 271,
"existing_alias_text": "SQL",
"input_term": "SQL",
"matched_canonical": {
"category_id": 6,
"display_name": "SQL",
"id": 101,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "LANGUAGE",
"slug": "sql",
"sub_category_id": 97,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "alias"
},
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 67,
"existing_alias_text": "Python",
"input_term": "Python",
"matched_canonical": {
"category_id": 6,
"display_name": "Python",
"id": 5,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "LANGUAGE",
"slug": "python",
"sub_category_id": 96,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "alias"
}
],
"candidate_roles": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
},
{
"display_name": "Drupal Dev",
"id": 228,
"rationale": null,
"role_archetype": "Engineering",
"slug": "drupal-dev",
"source": "db"
},
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
},
{
"display_name": "Cloud Architect",
"id": 9,
"rationale": null,
"role_archetype": null,
"slug": "cloud-architect",
"source": "db"
},
{
"display_name": "Pega Developer",
"id": 24,
"rationale": null,
"role_archetype": null,
"slug": "pega-developer",
"source": "db"
},
{
"display_name": "Engineering Manager",
"id": 121,
"rationale": null,
"role_archetype": null,
"slug": "engineering-manager",
"source": "db"
},
{
"display_name": "Cloud Security Engineer",
"id": 23,
"rationale": null,
"role_archetype": null,
"slug": "cloud-security-engineer",
"source": "db"
},
{
"display_name": "Backend Developer",
"id": 1,
"rationale": null,
"role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
"slug": "backend-engineer",
"source": "db"
},
{
"display_name": "Fullstack Developer",
"id": 15,
"rationale": null,
"role_archetype": null,
"slug": "full-stack-engineer",
"source": "db"
},
{
"display_name": "Fullstack Developer",
"id": 435,
"rationale": null,
"role_archetype": "Engineering",
"slug": "fullstack-developer",
"source": "db"
},
{
"display_name": "Cyber Security Engineer",
"id": 5,
"rationale": null,
"role_archetype": null,
"slug": "cybersecurity-engineer",
"source": "db"
},
{
"display_name": "AR/VR Engineer",
"id": 8,
"rationale": null,
"role_archetype": null,
"slug": "ar-vr-engineer",
"source": "db"
},
{
"display_name": "Python Backend Developer",
"id": 80,
"rationale": null,
"role_archetype": "Engineering",
"slug": "python-backend-developer",
"source": "db"
}
],
"chosen_role": {
"display_name": "Data Engineer",
"id": 2,
"rationale": "Exact alias hit on data-engineer (1.0) \u2014 no other alias at this confidence; skill_top data-engineer 0.33 does not contradict",
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Data Warehouses",
"id": 22,
"rationale": "Managed analytical storage and compute platforms used for curated datasets, reporting, and downstream analytics. These systems are central to data modeling, performance tuning, and cost-aware query design.",
"slug": "cloud-data-warehouses",
"source": "db"
},
"input_skill": "BigQuery",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Composer and Dependency Management",
"id": 355,
"rationale": "Managing Drupal project dependencies, package constraints, and update compatibility through Composer. This is a coherent cluster because Drupal development commonly depends on coordinated package versions and patch application.",
"slug": "composer-and-dependency-management",
"source": "db"
},
"input_skill": "Composer",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Drupal Dev",
"id": 228,
"rationale": null,
"role_archetype": "Engineering",
"slug": "drupal-dev",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Workflow Orchestration for ML Pipelines",
"id": 54,
"rationale": "Workflow engines used to coordinate training, evaluation, deployment, and retraining jobs. This cluster covers dependencies, retries, scheduling, and pipeline composition for ML lifecycle automation.",
"slug": "workflow-orchestration-for-ml-pipelines",
"source": "db"
},
"input_skill": "Airflow",
"llm_role": null,
"roles_from_db": [
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Storage and Data Services",
"id": 144,
"rationale": "Cloud-native storage and managed data services used to place workloads, choose durability tiers, and define platform boundaries. This is a coherent cluster because architects evaluate storage fit, access patterns, and managed service tradeoffs.",
"slug": "cloud-storage-and-data-services",
"source": "db"
},
"input_skill": "Google Cloud Storage",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Cloud Architect",
"id": 9,
"rationale": null,
"role_archetype": null,
"slug": "cloud-architect",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Storage and File Formats",
"id": 35,
"rationale": "Object storage and data file formats used as the physical substrate for data movement and lake-style analytics. Data engineers need these to manage landing zones, partitioned datasets, and efficient interchange.",
"slug": "cloud-storage-and-file-formats",
"source": "db"
},
"input_skill": "Google Cloud Storage",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "React Frontend Development",
"id": 96,
"rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
"slug": "d_init_01",
"source": "db"
},
"input_skill": "Git",
"llm_role": null,
"roles_from_db": []
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Pega Programming Languages \u0026 DSLs",
"id": 267,
"rationale": "Programming languages and domain-specific languages used in Pega development.",
"slug": "pega-programming-languages-dsls",
"source": "db"
},
"input_skill": "SQL",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Pega Developer",
"id": 24,
"rationale": null,
"role_archetype": null,
"slug": "pega-developer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages \u0026 DSLs",
"id": 475,
"rationale": "Oversee and guide the selection and effective use of programming and domain\u2010specific languages in software projects.",
"slug": "programming-languages-dsls",
"source": "db"
},
"input_skill": "SQL",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Engineering Manager",
"id": 121,
"rationale": null,
"role_archetype": null,
"slug": "engineering-manager",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages for Data Work",
"id": 21,
"rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
"slug": "programming-languages-for-data-work",
"source": "db"
},
"input_skill": "SQL",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Security Scripting \u0026 DSL Languages",
"id": 248,
"rationale": "Proficiency in programming and domain-specific languages used to automate and script cloud security controls.",
"slug": "cloud-security-scripting-dsl-languages",
"source": "db"
},
"input_skill": "Python",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Cloud Security Engineer",
"id": 23,
"rationale": null,
"role_archetype": null,
"slug": "cloud-security-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages",
"id": 1,
"rationale": "Primary implementation languages used to build client and server feature code. Full stack engineers need enough fluency to move across layers and implement product behavior end to end.",
"slug": "programming-languages",
"source": "db"
},
"input_skill": "Python",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Backend Developer",
"id": 1,
"rationale": null,
"role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
"slug": "backend-engineer",
"source": "db"
},
{
"display_name": "Fullstack Developer",
"id": 15,
"rationale": null,
"role_archetype": null,
"slug": "full-stack-engineer",
"source": "db"
},
{
"display_name": "Fullstack Developer",
"id": 435,
"rationale": null,
"role_archetype": "Engineering",
"slug": "fullstack-developer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages \u0026 DSLs",
"id": 475,
"rationale": "Oversee and guide the selection and effective use of programming and domain\u2010specific languages in software projects.",
"slug": "programming-languages-dsls",
"source": "db"
},
"input_skill": "Python",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Engineering Manager",
"id": 121,
"rationale": null,
"role_archetype": null,
"slug": "engineering-manager",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages and Scripting",
"id": 59,
"rationale": "Languages used to write security automation, analysis scripts, detection logic, and remediation helpers. This is the primary implementation surface for a cybersecurity engineer across tooling and response workflows.",
"slug": "programming-languages-and-scripting",
"source": "db"
},
"input_skill": "Python",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Cyber Security Engineer",
"id": 5,
"rationale": null,
"role_archetype": null,
"slug": "cybersecurity-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages for Data Work",
"id": 21,
"rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
"slug": "programming-languages-for-data-work",
"source": "db"
},
"input_skill": "Python",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages for ML Systems",
"id": 39,
"rationale": "Languages used to build training code, inference services, evaluation jobs, and ML glue code. This is the primary implementation surface for ML engineers across experimentation and productionization.",
"slug": "programming-languages-for-ml-systems",
"source": "db"
},
"input_skill": "Python",
"llm_role": null,
"roles_from_db": [
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages for XR",
"id": 97,
"rationale": "Primary implementation languages used to build immersive client features, interaction logic, and device-specific runtime behavior. This is the core coding surface for AR/VR experiences.",
"slug": "programming-languages-for-xr",
"source": "db"
},
"input_skill": "Python",
"llm_role": null,
"roles_from_db": [
{
"display_name": "AR/VR Engineer",
"id": 8,
"rationale": null,
"role_archetype": null,
"slug": "ar-vr-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Python Programming",
"id": 290,
"rationale": "Core Python language skills used to implement backend business logic, request handlers, integrations, and service internals. This is the primary coding surface for the role.",
"slug": "python-programming",
"source": "db"
},
"input_skill": "Python",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Python Backend Developer",
"id": 80,
"rationale": null,
"role_archetype": "Engineering",
"slug": "python-backend-developer",
"source": "db"
}
]
}
],
"input_final_skills": [
"BigQuery",
"Cloud Data Fusion",
"Composer",
"Airflow",
"Cloud Run",
"Google Cloud Storage",
"Git",
"Bitbucket",
"Cloud Build",
"SQL",
"Python",
"ETL"
],
"input_llm_skills": [
"BigQuery",
"Cloud Data Fusion",
"Composer",
"Airflow",
"Cloud Run",
"Google Cloud Storage",
"Git",
"Bitbucket",
"Cloud Build",
"SQL",
"Python",
"ETL"
],
"new_aliases_persisted": 0,
"run_id": "519de9f6-c0e1-4fae-8508-3a9dfd6a185c",
"skills_detail": [
{
"aliases_in_db": [
{
"alias_text": "BigQuery",
"alias_type": "CANONICAL",
"id": 300,
"is_primary": true,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 11,
"display_name": "BigQuery",
"id": 106,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "CLOUD_SERVICE",
"slug": "bigquery",
"sub_category_id": 118,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Data Warehouses",
"id": 22,
"rationale": "Managed analytical storage and compute platforms used for curated datasets, reporting, and downstream analytics. These systems are central to data modeling, performance tuning, and cost-aware query design.",
"slug": "cloud-data-warehouses",
"source": "db"
},
"input_skill": "BigQuery",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
}
],
"input_skill": "BigQuery",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [],
"canonical": null,
"dimensions": [],
"input_skill": "Cloud Data Fusion",
"matched_via": null,
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": {
"derived": {
"category": "Cloud Platforms",
"skill_nature": "PLATFORM",
"sub_category": "general",
"typical_lifespan": "MULTI_YEAR",
"version_strategy": "UNVERSIONED",
"volatility": "MEDIUM"
},
"enrichment": null,
"keep_log": [],
"locked_dimensions": [],
"merge_log": [],
"placed": null,
"relationships": null,
"skill_id": "cloud-data-fusion",
"split_log": [],
"typed": null,
"warnings": []
},
"source_tag": "llm",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "Composer",
"alias_type": "CANONICAL",
"id": 4603,
"is_primary": true,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 13,
"display_name": "Composer",
"id": 3126,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "TOOL",
"slug": "composer",
"sub_category_id": 2456,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Composer and Dependency Management",
"id": 355,
"rationale": "Managing Drupal project dependencies, package constraints, and update compatibility through Composer. This is a coherent cluster because Drupal development commonly depends on coordinated package versions and patch application.",
"slug": "composer-and-dependency-management",
"source": "db"
},
"input_skill": "Composer",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Drupal Dev",
"id": 228,
"rationale": null,
"role_archetype": "Engineering",
"slug": "drupal-dev",
"source": "db"
}
]
}
],
"input_skill": "Composer",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "Airflow",
"alias_type": "CANONICAL",
"id": 526,
"is_primary": true,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "airflow 2",
"alias_type": "VERSION",
"id": 2477,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "airflow-2",
"alias_type": "VERSION",
"id": 2478,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "airflow2",
"alias_type": "VERSION",
"id": 2476,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "airflow2.x",
"alias_type": "VERSION",
"id": 2479,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "apache airflow 2",
"alias_type": "VERSION",
"id": 2480,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 13,
"display_name": "Airflow",
"id": 265,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "TOOL",
"slug": "airflow",
"sub_category_id": 130,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Workflow Orchestration for ML Pipelines",
"id": 54,
"rationale": "Workflow engines used to coordinate training, evaluation, deployment, and retraining jobs. This cluster covers dependencies, retries, scheduling, and pipeline composition for ML lifecycle automation.",
"slug": "workflow-orchestration-for-ml-pipelines",
"source": "db"
},
"input_skill": "Airflow",
"llm_role": null,
"roles_from_db": [
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
}
]
}
],
"input_skill": "Airflow",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [],
"canonical": null,
"dimensions": [],
"input_skill": "Cloud Run",
"matched_via": null,
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": {
"derived": {
"category": "Cloud Platforms",
"skill_nature": "PLATFORM",
"sub_category": "general",
"typical_lifespan": "MULTI_YEAR",
"version_strategy": "UNVERSIONED",
"volatility": "MEDIUM"
},
"enrichment": null,
"keep_log": [],
"locked_dimensions": [],
"merge_log": [],
"placed": null,
"relationships": null,
"skill_id": "cloud-run",
"split_log": [],
"typed": null,
"warnings": []
},
"source_tag": "llm",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "Google Cloud Storage",
"alias_type": "CANONICAL",
"id": 380,
"is_primary": true,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 11,
"display_name": "Google Cloud Storage",
"id": 171,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "CLOUD_SERVICE",
"slug": "google-cloud-storage",
"sub_category_id": 120,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Storage and Data Services",
"id": 144,
"rationale": "Cloud-native storage and managed data services used to place workloads, choose durability tiers, and define platform boundaries. This is a coherent cluster because architects evaluate storage fit, access patterns, and managed service tradeoffs.",
"slug": "cloud-storage-and-data-services",
"source": "db"
},
"input_skill": "Google Cloud Storage",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Cloud Architect",
"id": 9,
"rationale": null,
"role_archetype": null,
"slug": "cloud-architect",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Storage and File Formats",
"id": 35,
"rationale": "Object storage and data file formats used as the physical substrate for data movement and lake-style analytics. Data engineers need these to manage landing zones, partitioned datasets, and efficient interchange.",
"slug": "cloud-storage-and-file-formats",
"source": "db"
},
"input_skill": "Google Cloud Storage",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
}
],
"input_skill": "Google Cloud Storage",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "Git",
"alias_type": "CANONICAL",
"id": 1613,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 13,
"display_name": "Git",
"id": 1002,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "TOOL",
"slug": "git",
"sub_category_id": 730,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "React Frontend Development",
"id": 96,
"rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
"slug": "d_init_01",
"source": "db"
},
"input_skill": "Git",
"llm_role": null,
"roles_from_db": []
}
],
"input_skill": "Git",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [],
"canonical": null,
"dimensions": [],
"input_skill": "Bitbucket",
"matched_via": null,
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": {
"derived": {
"category": "DevOps Tools",
"skill_nature": "TOOL",
"sub_category": "general",
"typical_lifespan": "MULTI_YEAR",
"version_strategy": "UNVERSIONED",
"volatility": "MEDIUM"
},
"enrichment": null,
"keep_log": [],
"locked_dimensions": [],
"merge_log": [],
"placed": null,
"relationships": null,
"skill_id": "bitbucket",
"split_log": [],
"typed": null,
"warnings": []
},
"source_tag": "llm",
"was_in_llm_skills": true
},
{
"aliases_in_db": [],
"canonical": null,
"dimensions": [],
"input_skill": "Cloud Build",
"matched_via": null,
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": {
"derived": {
"category": "Cloud Platforms",
"skill_nature": "PLATFORM",
"sub_category": "general",
"typical_lifespan": "MULTI_YEAR",
"version_strategy": "UNVERSIONED",
"volatility": "MEDIUM"
},
"enrichment": null,
"keep_log": [],
"locked_dimensions": [],
"merge_log": [],
"placed": null,
"relationships": null,
"skill_id": "cloud-build",
"split_log": [],
"typed": null,
"warnings": []
},
"source_tag": "llm",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "SQL",
"alias_type": "CANONICAL",
"id": 271,
"is_primary": true,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 6,
"display_name": "SQL",
"id": 101,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "LANGUAGE",
"slug": "sql",
"sub_category_id": 97,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Pega Programming Languages \u0026 DSLs",
"id": 267,
"rationale": "Programming languages and domain-specific languages used in Pega development.",
"slug": "pega-programming-languages-dsls",
"source": "db"
},
"input_skill": "SQL",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Pega Developer",
"id": 24,
"rationale": null,
"role_archetype": null,
"slug": "pega-developer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages \u0026 DSLs",
"id": 475,
"rationale": "Oversee and guide the selection and effective use of programming and domain\u2010specific languages in software projects.",
"slug": "programming-languages-dsls",
"source": "db"
},
"input_skill": "SQL",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Engineering Manager",
"id": 121,
"rationale": null,
"role_archetype": null,
"slug": "engineering-manager",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages for Data Work",
"id": 21,
"rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
"slug": "programming-languages-for-data-work",
"source": "db"
},
"input_skill": "SQL",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
}
],
"input_skill": "SQL",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "Python",
"alias_type": "CANONICAL",
"id": 67,
"is_primary": true,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "Python 2",
"alias_type": "VERSION",
"id": 72,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "Python 2.x",
"alias_type": "VERSION",
"id": 74,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "Python 3",
"alias_type": "VERSION",
"id": 73,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "Python 3.10",
"alias_type": "VERSION",
"id": 76,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "Python 3.11",
"alias_type": "VERSION",
"id": 77,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "Python 3.12",
"alias_type": "VERSION",
"id": 78,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "Python 3.x",
"alias_type": "VERSION",
"id": 75,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "py",
"alias_type": "VERSION",
"id": 2183,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "py2",
"alias_type": "VERSION",
"id": 68,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "py3",
"alias_type": "VERSION",
"id": 69,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "python 3",
"alias_type": "VERSION",
"id": 2186,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "python 3.x",
"alias_type": "VERSION",
"id": 2849,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "python2",
"alias_type": "VERSION",
"id": 70,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "python3",
"alias_type": "VERSION",
"id": 71,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "python3.x",
"alias_type": "VERSION",
"id": 2848,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 6,
"display_name": "Python",
"id": 5,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "LANGUAGE",
"slug": "python",
"sub_category_id": 96,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Security Scripting \u0026 DSL Languages",
"id": 248,
"rationale": "Proficiency in programming and domain-specific languages used to automate and script cloud security controls.",
"slug": "cloud-security-scripting-dsl-languages",
"source": "db"
},
"input_skill": "Python",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Cloud Security Engineer",
"id": 23,
"rationale": null,
"role_archetype": null,
"slug": "cloud-security-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages",
"id": 1,
"rationale": "Primary implementation languages used to build client and server feature code. Full stack engineers need enough fluency to move across layers and implement product behavior end to end.",
"slug": "programming-languages",
"source": "db"
},
"input_skill": "Python",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Backend Developer",
"id": 1,
"rationale": null,
"role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
"slug": "backend-engineer",
"source": "db"
},
{
"display_name": "Fullstack Developer",
"id": 15,
"rationale": null,
"role_archetype": null,
"slug": "full-stack-engineer",
"source": "db"
},
{
"display_name": "Fullstack Developer",
"id": 435,
"rationale": null,
"role_archetype": "Engineering",
"slug": "fullstack-developer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages \u0026 DSLs",
"id": 475,
"rationale": "Oversee and guide the selection and effective use of programming and domain\u2010specific languages in software projects.",
"slug": "programming-languages-dsls",
"source": "db"
},
"input_skill": "Python",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Engineering Manager",
"id": 121,
"rationale": null,
"role_archetype": null,
"slug": "engineering-manager",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages and Scripting",
"id": 59,
"rationale": "Languages used to write security automation, analysis scripts, detection logic, and remediation helpers. This is the primary implementation surface for a cybersecurity engineer across tooling and response workflows.",
"slug": "programming-languages-and-scripting",
"source": "db"
},
"input_skill": "Python",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Cyber Security Engineer",
"id": 5,
"rationale": null,
"role_archetype": null,
"slug": "cybersecurity-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages for Data Work",
"id": 21,
"rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
"slug": "programming-languages-for-data-work",
"source": "db"
},
"input_skill": "Python",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages for ML Systems",
"id": 39,
"rationale": "Languages used to build training code, inference services, evaluation jobs, and ML glue code. This is the primary implementation surface for ML engineers across experimentation and productionization.",
"slug": "programming-languages-for-ml-systems",
"source": "db"
},
"input_skill": "Python",
"llm_role": null,
"roles_from_db": [
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages for XR",
"id": 97,
"rationale": "Primary implementation languages used to build immersive client features, interaction logic, and device-specific runtime behavior. This is the core coding surface for AR/VR experiences.",
"slug": "programming-languages-for-xr",
"source": "db"
},
"input_skill": "Python",
"llm_role": null,
"roles_from_db": [
{
"display_name": "AR/VR Engineer",
"id": 8,
"rationale": null,
"role_archetype": null,
"slug": "ar-vr-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Python Programming",
"id": 290,
"rationale": "Core Python language skills used to implement backend business logic, request handlers, integrations, and service internals. This is the primary coding surface for the role.",
"slug": "python-programming",
"source": "db"
},
"input_skill": "Python",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Python Backend Developer",
"id": 80,
"rationale": null,
"role_archetype": "Engineering",
"slug": "python-backend-developer",
"source": "db"
}
]
}
],
"input_skill": "Python",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [],
"canonical": null,
"dimensions": [],
"input_skill": "ETL",
"matched_via": null,
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": {
"derived": {
"category": "Data Engineering Tools",
"skill_nature": "CONCEPT",
"sub_category": "general",
"typical_lifespan": "MULTI_YEAR",
"version_strategy": "UNVERSIONED",
"volatility": "MEDIUM"
},
"enrichment": null,
"keep_log": [],
"locked_dimensions": [],
"merge_log": [],
"placed": null,
"relationships": null,
"skill_id": "etl",
"split_log": [],
"typed": null,
"warnings": []
},
"source_tag": "llm",
"was_in_llm_skills": true
}
],
"unmatched_skills": [
"Cloud Data Fusion",
"Cloud Run",
"Bitbucket",
"Cloud Build",
"ETL"
]
}
API 3 — final-role-output
{
"chosen_role": {
"display_name": "Data Engineer",
"id": 2,
"rationale": "Exact alias hit on data-engineer (1.0) \u2014 no other alias at this confidence; skill_top data-engineer 0.33 does not contradict",
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
},
"chosen_role_resolution": "in_db",
"final_input_skills": [
{
"skill": "BigQuery",
"tag": "in_db"
},
{
"skill": "Cloud Data Fusion",
"tag": "new"
},
{
"skill": "Composer",
"tag": "in_db"
},
{
"skill": "Airflow",
"tag": "in_db"
},
{
"skill": "Cloud Run",
"tag": "new"
},
{
"skill": "Google Cloud Storage",
"tag": "in_db"
},
{
"skill": "Git",
"tag": "in_db"
},
{
"skill": "Bitbucket",
"tag": "new"
},
{
"skill": "Cloud Build",
"tag": "new"
},
{
"skill": "SQL",
"tag": "in_db"
},
{
"skill": "Python",
"tag": "in_db"
},
{
"skill": "ETL",
"tag": "new"
}
],
"llm_cost_api1_usd": null,
"llm_cost_api2_usd": null,
"llm_cost_api3_usd": null,
"llm_cost_total_usd": null,
"persistence": {
"items": [
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Data Warehouses",
"id": 22,
"rationale": "Managed analytical storage and compute platforms used for curated datasets, reporting, and downstream analytics. These systems are central to data modeling, performance tuning, and cost-aware query design.",
"slug": "cloud-data-warehouses",
"source": "db"
},
"dimension_id": 22,
"input_skill": "BigQuery",
"llm_role": null,
"matched_chosen_role": true,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
"role_dimension_saved": true,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 106,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Composer and Dependency Management",
"id": 355,
"rationale": "Managing Drupal project dependencies, package constraints, and update compatibility through Composer. This is a coherent cluster because Drupal development commonly depends on coordinated package versions and patch application.",
"slug": "composer-and-dependency-management",
"source": "db"
},
"dimension_id": 355,
"input_skill": "Composer",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Drupal Dev",
"id": 228,
"rationale": null,
"role_archetype": "Engineering",
"slug": "drupal-dev",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 3126,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Workflow Orchestration for ML Pipelines",
"id": 54,
"rationale": "Workflow engines used to coordinate training, evaluation, deployment, and retraining jobs. This cluster covers dependencies, retries, scheduling, and pipeline composition for ML lifecycle automation.",
"slug": "workflow-orchestration-for-ml-pipelines",
"source": "db"
},
"dimension_id": 54,
"input_skill": "Airflow",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 265,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Storage and Data Services",
"id": 144,
"rationale": "Cloud-native storage and managed data services used to place workloads, choose durability tiers, and define platform boundaries. This is a coherent cluster because architects evaluate storage fit, access patterns, and managed service tradeoffs.",
"slug": "cloud-storage-and-data-services",
"source": "db"
},
"dimension_id": 144,
"input_skill": "Google Cloud Storage",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Cloud Architect",
"id": 9,
"rationale": null,
"role_archetype": null,
"slug": "cloud-architect",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 171,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Storage and File Formats",
"id": 35,
"rationale": "Object storage and data file formats used as the physical substrate for data movement and lake-style analytics. Data engineers need these to manage landing zones, partitioned datasets, and efficient interchange.",
"slug": "cloud-storage-and-file-formats",
"source": "db"
},
"dimension_id": 35,
"input_skill": "Google Cloud Storage",
"llm_role": null,
"matched_chosen_role": true,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
"role_dimension_saved": true,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 171,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "React Frontend Development",
"id": 96,
"rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
"slug": "d_init_01",
"source": "db"
},
"dimension_id": 96,
"input_skill": "Git",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [],
"skill_dimension_saved": true,
"skill_id": 1002,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Pega Programming Languages \u0026 DSLs",
"id": 267,
"rationale": "Programming languages and domain-specific languages used in Pega development.",
"slug": "pega-programming-languages-dsls",
"source": "db"
},
"dimension_id": 267,
"input_skill": "SQL",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Pega Developer",
"id": 24,
"rationale": null,
"role_archetype": null,
"slug": "pega-developer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 101,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages \u0026 DSLs",
"id": 475,
"rationale": "Oversee and guide the selection and effective use of programming and domain\u2010specific languages in software projects.",
"slug": "programming-languages-dsls",
"source": "db"
},
"dimension_id": 475,
"input_skill": "SQL",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Engineering Manager",
"id": 121,
"rationale": null,
"role_archetype": null,
"slug": "engineering-manager",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 101,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages for Data Work",
"id": 21,
"rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
"slug": "programming-languages-for-data-work",
"source": "db"
},
"dimension_id": 21,
"input_skill": "SQL",
"llm_role": null,
"matched_chosen_role": true,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
"role_dimension_saved": true,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 101,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Security Scripting \u0026 DSL Languages",
"id": 248,
"rationale": "Proficiency in programming and domain-specific languages used to automate and script cloud security controls.",
"slug": "cloud-security-scripting-dsl-languages",
"source": "db"
},
"dimension_id": 248,
"input_skill": "Python",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Cloud Security Engineer",
"id": 23,
"rationale": null,
"role_archetype": null,
"slug": "cloud-security-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 5,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages",
"id": 1,
"rationale": "Primary implementation languages used to build client and server feature code. Full stack engineers need enough fluency to move across layers and implement product behavior end to end.",
"slug": "programming-languages",
"source": "db"
},
"dimension_id": 1,
"input_skill": "Python",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Backend Developer",
"id": 1,
"rationale": null,
"role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
"slug": "backend-engineer",
"source": "db"
},
{
"display_name": "Fullstack Developer",
"id": 15,
"rationale": null,
"role_archetype": null,
"slug": "full-stack-engineer",
"source": "db"
},
{
"display_name": "Fullstack Developer",
"id": 435,
"rationale": null,
"role_archetype": "Engineering",
"slug": "fullstack-developer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 5,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages \u0026 DSLs",
"id": 475,
"rationale": "Oversee and guide the selection and effective use of programming and domain\u2010specific languages in software projects.",
"slug": "programming-languages-dsls",
"source": "db"
},
"dimension_id": 475,
"input_skill": "Python",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Engineering Manager",
"id": 121,
"rationale": null,
"role_archetype": null,
"slug": "engineering-manager",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 5,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages and Scripting",
"id": 59,
"rationale": "Languages used to write security automation, analysis scripts, detection logic, and remediation helpers. This is the primary implementation surface for a cybersecurity engineer across tooling and response workflows.",
"slug": "programming-languages-and-scripting",
"source": "db"
},
"dimension_id": 59,
"input_skill": "Python",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Cyber Security Engineer",
"id": 5,
"rationale": null,
"role_archetype": null,
"slug": "cybersecurity-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 5,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages for Data Work",
"id": 21,
"rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
"slug": "programming-languages-for-data-work",
"source": "db"
},
"dimension_id": 21,
"input_skill": "Python",
"llm_role": null,
"matched_chosen_role": true,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
"role_dimension_saved": true,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 5,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages for ML Systems",
"id": 39,
"rationale": "Languages used to build training code, inference services, evaluation jobs, and ML glue code. This is the primary implementation surface for ML engineers across experimentation and productionization.",
"slug": "programming-languages-for-ml-systems",
"source": "db"
},
"dimension_id": 39,
"input_skill": "Python",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 5,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages for XR",
"id": 97,
"rationale": "Primary implementation languages used to build immersive client features, interaction logic, and device-specific runtime behavior. This is the core coding surface for AR/VR experiences.",
"slug": "programming-languages-for-xr",
"source": "db"
},
"dimension_id": 97,
"input_skill": "Python",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "AR/VR Engineer",
"id": 8,
"rationale": null,
"role_archetype": null,
"slug": "ar-vr-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 5,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Python Programming",
"id": 290,
"rationale": "Core Python language skills used to implement backend business logic, request handlers, integrations, and service internals. This is the primary coding surface for the role.",
"slug": "python-programming",
"source": "db"
},
"dimension_id": 290,
"input_skill": "Python",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Python Backend Developer",
"id": 80,
"rationale": null,
"role_archetype": "Engineering",
"slug": "python-backend-developer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 5,
"skill_tag": "in_db",
"skipped_reason": null
}
],
"new_skills_created": 0,
"role_dimension_saved": 0,
"skill_dimension_saved": 0,
"skipped": 0
},
"planner_output": null,
"run_id": "519de9f6-c0e1-4fae-8508-3a9dfd6a185c"
}
LLM Calls
Every model call made for this run, in pipeline order. Click a card to see the model's response.