Pipeline run
216829da-da8a-4849-996b-1fd470c98354
Client output enrichment
v2 Skill cluster · Nature of work · AI index · Tech stack maturity · Evidence · KRA descriptionvocab breakdown (legacy)
Signals
Post-classification
Captured for admin review
1 POST /skills/extract-from-jd
2 POST /skills/extract-details
3 POST /skills/final-role-output
Data Engineer
domain · Data Engineering & Analytics CASE DOMAINslug: data-engineer · id: 2 · source: db
Domain=Data Engineering & Analytics; The JD is centered on Spark, Databricks, AWS, Airflow, big data, and lakehouse/data warehousing work, which most closely matches Data Engineer.
Matched skills
Matched dimensions
Matched KRAs
Resolution:
in_db
— role exists in library; skill↔dim and role↔dim links saved when applicable.
Job description
Description JOB DESCRIPTION We have an exciting and rewarding opportunity for you to take your software engineering career to the next level. Job Summary As a Software Engineer II at JPMorgan Chase within the Corporate technology, you design and deliver trusted technology products in a secure, stable, and scalable way. You are responsible for implementing critical solutions across multiple technical areas to support the firm’s business objectives. Job Responsibilities • Execute software solutions, design, development, and technical troubleshooting to solve complex problems. • Create secure, high-quality production code and maintain algorithms that run synchronously with systems. • Produce architecture and design artifacts for complex applications, ensuring design constraints are met. • Gather, analyze, and develop visualizations and reporting from large, diverse data sets to improve applications and systems. • Identify hidden problems and patterns in data to drive improvements in coding hygiene and system architecture. • Contribute to software engineering communities of practice and events exploring new and emerging technologies. Required Qualifications, Capabilities, And Skills • Formal training or certification on software engineering concepts and 2+ years applied experience • At least one year of practical experience with Spark, SQL, Databricks, and AWS cloud ecosystem. • Expertise in Apache NiFi, Lakehouse or Delta Lake architectures, system design, application development, testing, and operational stability. • Strong programming skills in PySpark and SparkSQL. • Proficient in orchestration using Airflow. • In-depth knowledge of Big Data and data warehousing concepts. • Experience with CI/CD processes. Preferred Qualifications, Capabilities, And Skills • Familiarity with Snowflake, Terraform, and large language models. • Exposure to cloud technologies such as AWS Glue, S3, SQS, SNS, Lambda. • AWS certifications such as Solutions Architect Associate, Developer Associate, Data Analytics Specialty, or Databricks certification.
Skills from this JD
Each row merges API 1 extraction, API 2 library match / v3 orchestration (dimensions + locked dims), and API 3 persistence tags.
Aliases — catalog
- Apache Spark (CANONICAL)
- apache spark 3 (VERSION)
- spark (VERSION)
- spark 3 (VERSION)
- spark 3.x (VERSION)
- spark3 (VERSION)
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Framework
- Sub-category
- Distributed Data Processing Framework
- Vendor
- Apache Software Foundation
- License
- apache_2
- Year introduced
- 2010
- Confidence
- 0.94
- Version strategy
- SEPARATE_ENTITY
- Version tag
- 3.x
Maturity reasoning: Apache Spark appears in many data engineering JDs and remains a standard for distributed ETL/ELT; its GitHub and vendor ecosystem activity stay strong, with Databricks and cloud platforms still promoting it.
Skill profile (library / DB)
- Skill nature
- FRAMEWORK
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 5
- Sub-category id
- 1021
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
ETL and ELT Tooling Catalog dimension db id 24
Library dimension (catalog)
Roles linked in library: Data Engineer
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
ETL and ELT Tooling
etl-and-elt-tooling
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved |
Aliases — catalog
- SQL (CANONICAL) primary
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Language
- Sub-category
- Query Language
- Vendor
- ANSI
- License
- unknown
- Year introduced
- 1974
- Confidence
- 0.99
- Version strategy
- NOT_APPLICABLE
Maturity reasoning: SQL appears in a large share of data, backend, and analytics job descriptions and remains the default query language for PostgreSQL, MySQL, and cloud warehouses like Snowflake/BigQuery.
Skill profile (library / DB)
- Skill nature
- LANGUAGE
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 6
- Sub-category id
- 97
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
Pega Programming Languages & DSLs Catalog dimension db id 267
Library dimension (catalog)
Roles linked in library: Pega Developer
-
Programming Languages & DSLs Catalog dimension db id 475
Library dimension (catalog)
Roles linked in library: Engineering Manager
-
Programming Languages for Data Work Catalog dimension db id 21
Library dimension (catalog)
Roles linked in library: Data Engineer
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
Pega Programming Languages & DSLs
pega-programming-languages-dsls
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Programming Languages & DSLs
programming-languages-dsls
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Programming Languages for Data Work
programming-languages-for-data-work
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved |
Aliases — catalog
- Databricks (CANONICAL)
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Platform
- Sub-category
- Data Analytics Platform
- Vendor
- Databricks, Inc.
- License
- other_open
- Year introduced
- 2013
- Confidence
- 0.97
- Version strategy
- NOT_APPLICABLE
Maturity reasoning: Databricks appears frequently in data engineering and analytics job postings, especially alongside Spark, Delta Lake, and lakehouse stacks; strong vendor adoption and broad enterprise usage signal mainstream demand.
Skill profile (library / DB)
- Skill nature
- PLATFORM
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 9
- Sub-category id
- 911
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
React Frontend Development Catalog dimension db id 96
Library dimension (catalog)
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
React Frontend Development
d_init_01
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
Aliases — catalog
- AWS (CANONICAL) primary
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Platform
- Sub-category
- Cloud Platform
- Vendor
- Amazon
- License
- other_open
- Year introduced
- 2006
- Confidence
- 0.99
- Version strategy
- NOT_APPLICABLE
Maturity reasoning: AWS is a hiring-pipeline staple: it appears in a large share of cloud/DevOps job descriptions and dominates public cloud market share, with broad certification and vendor ecosystem support.
Skill profile (library / DB)
- Skill nature
- PLATFORM
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 9
- Sub-category id
- 46
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
Cloud Platforms Catalog dimension db id 20
Library dimension (catalog)
Roles linked in library: .NET Backend Developer, Backend Developer, Cyber Security Engineer, Data Engineer, DevOps Engineer, Fullstack Developer, Go Backend Developer, Java Backend Developer, Kotlin Backend Developer, ML Engineer, MLOps Engineer, Node.js Backend Developer, Python Backend Developer, Scala Backend Developer
-
Cloud Platforms for AI Deployment Catalog dimension db id 211
Library dimension (catalog)
Roles linked in library: AI Engineer
-
Cloud Provider Platforms Catalog dimension db id 131
Library dimension (catalog)
Roles linked in library: Cloud Architect, Cloud Security Engineer
-
Cloud Security Posture Tools Catalog dimension db id 64
Library dimension (catalog)
Roles linked in library: Cloud Security Engineer, Cyber Security Engineer
-
Vendor Product Families Catalog dimension db id 477
Library dimension (catalog)
Roles linked in library: Engineering Manager
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
Cloud Platforms
cloud-platforms
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved |
|
Cloud Platforms for AI Deployment
cloud-platforms-for-ai-deployment
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Cloud Provider Platforms
cloud-provider-platforms
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Cloud Security Posture Tools
cloud-security-posture-tools
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Vendor Product Families
vendor-product-families
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
Skill enrichment (orchestrator / LLM)
No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).
- Category
- Data Engineering Tools
- Sub-category
- general
- Skill nature
- TOOL
- Volatility
- MEDIUM
- Typical lifespan
- MULTI_YEAR
- Version strategy
- UNVERSIONED
Aliases — catalog
- Lakehouse (CANONICAL)
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Architecture
- Sub-category
- Data Platform Architecture
- Confidence
- 0.90
- Version strategy
- NOT_APPLICABLE
Maturity reasoning: Lakehouse is increasingly listed in data-platform JDs and vendor docs (Databricks, Snowflake, Microsoft Fabric), but it is not yet as universal as core warehouse or lake skills.
Skill profile (library / DB)
- Skill nature
- PATTERN
- Volatility
- EMERGING
- Typical lifespan
- EVERGREEN
- Category id
- 1
- Sub-category id
- 1026
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
React Frontend Development Catalog dimension db id 96
Library dimension (catalog)
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
React Frontend Development
d_init_01
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
Aliases — catalog
- Delta Lake (CANONICAL) primary
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Tool
- Sub-category
- Table Format Tool
- Vendor
- Databricks
- License
- apache_2
- Year introduced
- 2017
- Confidence
- 0.72
- Version strategy
- NOT_APPLICABLE
Maturity reasoning: Delta Lake appears frequently in data engineering JDs and cloud vendor docs, especially alongside Databricks/Spark for lakehouse stacks; it’s a common hiring-pipeline skill rather than a niche tool.
Skill profile (library / DB)
- Skill nature
- TOOL
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 13
- Sub-category id
- 1170
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
Model and Data Versioning Catalog dimension db id 48
Library dimension (catalog)
Roles linked in library: ML Engineer, MLOps Engineer
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
Model and Data Versioning
model-and-data-versioning
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
Aliases — catalog
- Apache Spark (CANONICAL)
- apache spark 3 (VERSION)
- spark (VERSION)
- spark 3 (VERSION)
- spark 3.x (VERSION)
- spark3 (VERSION)
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Framework
- Sub-category
- Distributed Data Processing Framework
- Vendor
- Apache Software Foundation
- License
- apache_2
- Year introduced
- 2010
- Confidence
- 0.94
- Version strategy
- SEPARATE_ENTITY
- Version tag
- 3.x
Maturity reasoning: Apache Spark appears in many data engineering JDs and remains a standard for distributed ETL/ELT; its GitHub and vendor ecosystem activity stay strong, with Databricks and cloud platforms still promoting it.
Skill profile (library / DB)
- Skill nature
- FRAMEWORK
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 5
- Sub-category id
- 1021
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
ETL and ELT Tooling Catalog dimension db id 24
Library dimension (catalog)
Roles linked in library: Data Engineer
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
ETL and ELT Tooling
etl-and-elt-tooling
|
— | — |
Skipped — no persistable v3 meta for new skill
skill_not_in_db_v3_proposed
|
Skill enrichment (orchestrator / LLM)
No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).
- Category
- Data Engineering Tools
- Sub-category
- general
- Skill nature
- LANGUAGE
- Volatility
- MEDIUM
- Typical lifespan
- MULTI_YEAR
- Version strategy
- UNVERSIONED
Aliases — catalog
- Airflow (CANONICAL) primary
- airflow 2 (VERSION)
- airflow-2 (VERSION)
- airflow2 (VERSION)
- airflow2.x (VERSION)
- apache airflow 2 (VERSION)
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Tool
- Sub-category
- Workflow Orchestration Tool
- Vendor
- Apache Software Foundation
- License
- apache_2
- Year introduced
- 2014
- Confidence
- 0.95
- Version strategy
- SEPARATE_ENTITY
- Version tag
- 2.x
Maturity reasoning: Apache Airflow appears in many data engineering job postings and is a common orchestration choice in production stacks; its GitHub activity and ecosystem remain strong, with no vendor sunset or clear replacement dominating JDs.
Skill profile (library / DB)
- Skill nature
- TOOL
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 13
- Sub-category id
- 130
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
Workflow Orchestration for ML Pipelines Catalog dimension db id 54
Library dimension (catalog)
Roles linked in library: ML Engineer, MLOps Engineer
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
Workflow Orchestration for ML Pipelines
workflow-orchestration-for-ml-pipelines
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
Skill enrichment (orchestrator / LLM)
No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).
- Category
- Concepts
- Sub-category
- general
- Skill nature
- CONCEPT
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Version strategy
- UNVERSIONED
Skill enrichment (orchestrator / LLM)
No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).
- Category
- Concepts
- Sub-category
- general
- Skill nature
- CONCEPT
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Version strategy
- UNVERSIONED
Aliases — catalog
- CI/CD (CANONICAL)
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Methodology
- Sub-category
- Ci Cd Process
- Confidence
- 0.93
- Version strategy
- NOT_APPLICABLE
Maturity reasoning: CI/CD appears in a large share of software engineering JDs and is a standard requirement across DevOps, platform, and backend roles; major vendors like GitHub, GitLab, and AWS all center product roadmaps on CI/CD pipelines.
Skill profile (library / DB)
- Skill nature
- METHODOLOGY
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 8
- Sub-category id
- 900
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
CI/CD Pipeline Platforms Catalog dimension db id 150
Library dimension (catalog)
Roles linked in library: DevOps Engineer
-
CI/CD for Machine Learning Catalog dimension db id 56
Library dimension (catalog)
Roles linked in library: ML Engineer
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
CI/CD Pipeline Platforms
ci-cd-pipeline-platforms
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
CI/CD for Machine Learning
ci-cd-for-machine-learning
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
Aliases — catalog
- Snowflake (CANONICAL) primary
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Platform
- Sub-category
- Data Cloud Platform
- Vendor
- Snowflake Inc.
- License
- proprietary
- Year introduced
- 2012
- Confidence
- 0.98
- Version strategy
- NOT_APPLICABLE
Maturity reasoning: Snowflake appears frequently in data/analytics job postings and is a standard cloud data warehouse platform alongside BigQuery and Redshift.
Skill profile (library / DB)
- Skill nature
- PLATFORM
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 9
- Sub-category id
- 113
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
Cloud Data Warehouses Catalog dimension db id 22
Library dimension (catalog)
Roles linked in library: Data Engineer
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
Cloud Data Warehouses
cloud-data-warehouses
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved |
Aliases — catalog
- Terraform (CANONICAL) primary
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Tool
- Sub-category
- Infrastructure As Code Tool
- Vendor
- HashiCorp
- License
- mpl
- Year introduced
- 2014
- Confidence
- 0.93
- Version strategy
- NOT_APPLICABLE
Maturity reasoning: Terraform is broadly listed in DevOps/SRE/cloud JDs and remains a standard IaC tool across AWS/Azure/GCP; HashiCorp’s ecosystem and widespread GitHub usage signal strong market adoption.
Skill profile (library / DB)
- Skill nature
- TOOL
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 13
- Sub-category id
- 191
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
Infrastructure & Security Automation Frameworks Catalog dimension db id 249
Library dimension (catalog)
Roles linked in library: Cloud Security Engineer
-
Infrastructure as Code Catalog dimension db id 132
Library dimension (catalog)
Roles linked in library: Cloud Architect, DevOps Engineer
-
Infrastructure as Code for ML Catalog dimension db id 57
Library dimension (catalog)
Roles linked in library: ML Engineer
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
Infrastructure & Security Automation Frameworks
infrastructure-security-automation-frameworks
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Infrastructure as Code
infrastructure-as-code
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Infrastructure as Code for ML
infrastructure-as-code-for-ml
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
Skill enrichment (orchestrator / LLM)
No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).
- Category
- Machine Learning Frameworks
- Sub-category
- general
- Skill nature
- CONCEPT
- Volatility
- FAST
- Typical lifespan
- SHORT_LIVED
- Version strategy
- VERSIONED
Skill enrichment (orchestrator / LLM)
No Stage 7 enrichment blob on this skill (orchestrator skipped enrichment).
- Category
- Cloud Platforms
- Sub-category
- general
- Skill nature
- PLATFORM
- Volatility
- MEDIUM
- Typical lifespan
- MULTI_YEAR
- Version strategy
- UNVERSIONED
Aliases — catalog
- Amazon S3 (CANONICAL) primary
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Service
- Sub-category
- Object Storage Service
- Vendor
- Amazon Web Services
- License
- proprietary
- Year introduced
- 2006
- Confidence
- 0.98
- Version strategy
- NOT_APPLICABLE
Maturity reasoning: Amazon S3 is a standard cloud storage service widely listed in job descriptions and core AWS certifications; it remains a default object-storage choice rather than a niche or sunset product.
Skill profile (library / DB)
- Skill nature
- CLOUD_SERVICE
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 11
- Sub-category id
- 120
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
Cloud Storage and Data Services Catalog dimension db id 144
Library dimension (catalog)
Roles linked in library: Cloud Architect
-
Cloud Storage and File Formats Catalog dimension db id 35
Library dimension (catalog)
Roles linked in library: Data Engineer
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
Cloud Storage and Data Services
cloud-storage-and-data-services
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Cloud Storage and File Formats
cloud-storage-and-file-formats
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved |
Aliases — catalog
- Amazon SQS (CANONICAL) primary
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Service
- Sub-category
- Message Queue Service
- Vendor
- Amazon
- License
- other_open
- Year introduced
- 2004
- Confidence
- 0.98
- Version strategy
- NOT_APPLICABLE
Maturity reasoning: Amazon SQS appears frequently in cloud/backend job descriptions and is a standard AWS managed queue for decoupled systems; it remains actively supported with no sunset signal.
Skill profile (library / DB)
- Skill nature
- CLOUD_SERVICE
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 11
- Sub-category id
- 56
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
Messaging and Background Jobs Catalog dimension db id 291
Library dimension (catalog)
Roles linked in library: PHP Backend Developer, Python Backend Developer, Ruby Backend Developer
-
Messaging and Event Streaming Catalog dimension db id 8
Library dimension (catalog)
Roles linked in library: Backend Developer, Data Engineer
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
Messaging and Background Jobs
messaging-and-background-jobs
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Messaging and Event Streaming
messaging-and-event-streaming
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved |
Aliases — catalog
- AWS SNS (CANONICAL) primary
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Service
- Sub-category
- Notification Service
- Vendor
- Amazon Web Services
- License
- proprietary
- Year introduced
- 2010
- Confidence
- 0.99
- Version strategy
- NOT_APPLICABLE
Maturity reasoning: Commonly listed in cloud/backend JDs for pub/sub and notifications; AWS continues to actively support SNS with broad ecosystem usage, indicating strong market demand.
Skill profile (library / DB)
- Skill nature
- CLOUD_SERVICE
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 11
- Sub-category id
- 2847
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
Asynchronous Messaging and Event Streaming Catalog dimension db id 297
Library dimension (catalog)
Roles linked in library: .NET Backend Developer, Go Backend Developer, Kotlin Backend Developer, Node.js Backend Developer, Scala Backend Developer
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
Asynchronous Messaging and Event Streaming
asynchronous-messaging-and-event-streaming
|
— | — |
Skipped — no persistable v3 meta for new skill
skill_not_in_db_v3_proposed
|
Aliases — catalog
- AWS Lambda (CANONICAL) primary
Context tags (catalog)
Stored enrichment (catalog DB)
- Category
- Service
- Sub-category
- Serverless Function Service
- Vendor
- Amazon Web Services
- License
- other_open
- Year introduced
- 2014
- Confidence
- 0.98
- Version strategy
- NOT_APPLICABLE
Maturity reasoning: Broadly listed in cloud/backend JDs and AWS’s serverless docs/marketplace; it’s a standard hiring-pipeline skill for event-driven apps and APIs.
Skill profile (library / DB)
- Skill nature
- CLOUD_SERVICE
- Volatility
- STABLE
- Typical lifespan
- EVERGREEN
- Category id
- 11
- Sub-category id
- 1704
- Extractable
- True
- Also category
- False
Dimensions (API 2 worklist)
-
Cloud Platforms Catalog dimension db id 20
Library dimension (catalog)
Roles linked in library: .NET Backend Developer, Backend Developer, Cyber Security Engineer, Data Engineer, DevOps Engineer, Fullstack Developer, Go Backend Developer, Java Backend Developer, Kotlin Backend Developer, ML Engineer, MLOps Engineer, Node.js Backend Developer, Python Backend Developer, Scala Backend Developer
-
Cloud Platforms & Hosting Providers Catalog dimension db id 278
Library dimension (catalog)
Roles linked in library: .NET Backend Developer, Kotlin Backend Developer, Scala Backend Developer, Web Developer
-
Cloud Platforms & Managed Services Catalog dimension db id 221
Library dimension (catalog)
Roles linked in library: Fullstack Developer, Go Backend Developer, Node.js Backend Developer
-
Cloud Platforms & Services Catalog dimension db id 296
Library dimension (catalog)
Roles linked in library: Python Backend Developer
API 3 link attempts (this skill)
| Dimension | Skill↔dim | Role↔dim | Outcome |
|---|---|---|---|
|
Cloud Platforms
cloud-platforms
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved |
|
Cloud Platforms & Hosting Providers
cloud-platforms-hosting-providers
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Cloud Platforms & Managed Services
cloud-platforms-managed-services
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
|
Cloud Platforms & Services
cloud-platforms-services
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
All API 3 persistence rows
Same grid as the skill-extractor “Persistence items” table: one row per (skill × dimension) work item.
| Skill | Tag | Dimension | Skill↔dim | Role↔dim | Outcome | Notes |
|---|---|---|---|---|---|---|
| Spark | in_db |
ETL and ELT Tooling
etl-and-elt-tooling
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved | |
| SQL | in_db |
Pega Programming Languages & DSLs
pega-programming-languages-dsls
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| SQL | in_db |
Programming Languages & DSLs
programming-languages-dsls
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| SQL | in_db |
Programming Languages for Data Work
programming-languages-for-data-work
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved | |
| Databricks | in_db |
React Frontend Development
d_init_01
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| AWS | in_db |
Cloud Platforms
cloud-platforms
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved | |
| AWS | in_db |
Cloud Platforms for AI Deployment
cloud-platforms-for-ai-deployment
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| AWS | in_db |
Cloud Provider Platforms
cloud-provider-platforms
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| AWS | in_db |
Cloud Security Posture Tools
cloud-security-posture-tools
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| AWS | in_db |
Vendor Product Families
vendor-product-families
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| Lakehouse | in_db |
React Frontend Development
d_init_01
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| Delta Lake | in_db |
Model and Data Versioning
model-and-data-versioning
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| PySpark | new |
ETL and ELT Tooling
etl-and-elt-tooling
|
— | — | Skipped — no persistable v3 meta for new skill | skill_not_in_db_v3_proposed |
| Airflow | in_db |
Workflow Orchestration for ML Pipelines
workflow-orchestration-for-ml-pipelines
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| CI/CD | in_db |
CI/CD Pipeline Platforms
ci-cd-pipeline-platforms
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| CI/CD | in_db |
CI/CD for Machine Learning
ci-cd-for-machine-learning
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| Snowflake | in_db |
Cloud Data Warehouses
cloud-data-warehouses
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved | |
| Terraform | in_db |
Infrastructure & Security Automation Frameworks
infrastructure-security-automation-frameworks
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| Terraform | in_db |
Infrastructure as Code
infrastructure-as-code
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| Terraform | in_db |
Infrastructure as Code for ML
infrastructure-as-code-for-ml
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| Amazon S3 | in_db |
Cloud Storage and Data Services
cloud-storage-and-data-services
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| Amazon S3 | in_db |
Cloud Storage and File Formats
cloud-storage-and-file-formats
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved | |
| Amazon SQS | in_db |
Messaging and Background Jobs
messaging-and-background-jobs
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| Amazon SQS | in_db |
Messaging and Event Streaming
messaging-and-event-streaming
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved | |
| Amazon SNS | new |
Asynchronous Messaging and Event Streaming
asynchronous-messaging-and-event-streaming
|
— | — | Skipped — no persistable v3 meta for new skill | skill_not_in_db_v3_proposed |
| AWS Lambda | in_db |
Cloud Platforms
cloud-platforms
|
✓ | ✓ | Existing dimension (library) · Role↔dimension saved | |
| AWS Lambda | in_db |
Cloud Platforms & Hosting Providers
cloud-platforms-hosting-providers
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| AWS Lambda | in_db |
Cloud Platforms & Managed Services
cloud-platforms-managed-services
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) | |
| AWS Lambda | in_db |
Cloud Platforms & Services
cloud-platforms-services
|
✓ | — | Existing dimension (library) · Role↔dimension skipped (dimension not under chosen role) |
Library artifacts (this run)
| Kind | Detail | DB id |
|---|---|---|
| canonical_skill_proposed | Apache NiFi | type=Data Engineering Tools subtype=general nature=TOOL lifespan=MULTI_YEAR | |
| canonical_skill_proposed | SparkSQL | type=Data Engineering Tools subtype=general nature=LANGUAGE lifespan=MULTI_YEAR | |
| canonical_skill_proposed | Big Data | type=Concepts subtype=general nature=CONCEPT lifespan=EVERGREEN | |
| canonical_skill_proposed | Data Warehousing | type=Concepts subtype=general nature=CONCEPT lifespan=EVERGREEN | |
| canonical_skill_proposed | Large Language Models | type=Machine Learning Frameworks subtype=general nature=CONCEPT lifespan=SHORT_LIVED | |
| canonical_skill_proposed | AWS Glue | type=Cloud Platforms subtype=general nature=PLATFORM lifespan=MULTI_YEAR | |
| dimension_skill_link_proposed | PySpark ↔ ETL and ELT Tooling | |
| role_dimension_link_proposed | Data Engineer ↔ ETL and ELT Tooling | |
| dimension_skill_link_proposed | Amazon SNS ↔ Asynchronous Messaging and Event Streaming |
nano JD Parser — gpt-4.1-nano click to toggle
Certifications
Show raw JSON
{
"JD_type": "pass",
"about_company": null,
"certifications": [
"AWS Solutions Architect Associate",
"AWS Developer Associate",
"AWS Data Analytics Specialty",
"Databricks certification"
],
"company_name": "JPMorgan Chase",
"ctc": null,
"domain": {
"primary": {
"aliases": [
"FinTech",
"Banking"
],
"domain": "Financial Services"
},
"secondary": null
},
"education": [],
"experience": {
"max": null,
"min": 2,
"raw": "2+ years applied experience"
},
"job_locations": [],
"role": "Software Engineer II",
"role_aliases": [
"Software Engineer",
"SWE",
"Software Developer"
],
"role_archetype": "Engineering",
"roles_and_responsibilities": [
{
"bullet_count": 6,
"heading": "Job Responsibilities",
"heading_was_present": true,
"source_marker": {
"first_5_words": "\u2022 Execute software solutions, design,",
"last_5_words": "new and emerging technologies."
},
"text": "\u2022 Execute software solutions, design, development, and technical troubleshooting to solve complex problems.\n\u2022 Create secure, high-quality production code and maintain algorithms that run synchronously with systems.\n\u2022 Produce architecture and design artifacts for complex applications, ensuring design constraints are met.\n\u2022 Gather, analyze, and develop visualizations and reporting from large, diverse data sets to improve applications and systems.\n\u2022 Identify hidden problems and patterns in data to drive improvements in coding hygiene and system architecture.\n\u2022 Contribute to software engineering communities of practice and events exploring new and emerging technologies.",
"word_count": 83
},
{
"bullet_count": 7,
"heading": "Required Qualifications, Capabilities, And Skills",
"heading_was_present": true,
"source_marker": {
"first_5_words": "\u2022 Formal training or certification",
"last_5_words": "and CI/CD processes."
},
"text": "\u2022 Formal training or certification on software engineering concepts and 2+ years applied experience\n\u2022 At least one year of practical experience with Spark, SQL, Databricks, and AWS cloud ecosystem.\n\u2022 Expertise in Apache NiFi, Lakehouse or Delta Lake architectures, system design, application development, testing, and operational stability.\n\u2022 Strong programming skills in PySpark and SparkSQL.\n\u2022 Proficient in orchestration using Airflow.\n\u2022 In-depth knowledge of Big Data and data warehousing concepts.\n\u2022 Experience with CI/CD processes.",
"word_count": 85
},
{
"bullet_count": 3,
"heading": "Preferred Qualifications, Capabilities, And Skills",
"heading_was_present": true,
"source_marker": {
"first_5_words": "\u2022 Familiarity with Snowflake, Terraform,",
"last_5_words": "or Databricks certification."
},
"text": "\u2022 Familiarity with Snowflake, Terraform, and large language models.\n\u2022 Exposure to cloud technologies such as AWS Glue, S3, SQS, SNS, Lambda.\n\u2022 AWS certifications such as Solutions Architect Associate, Developer Associate, Data Analytics Specialty, or Databricks certification.",
"word_count": 47
}
],
"urls": []
}
API 1 — extract-from-jd click to toggle
{
"final_skills": [
{
"is_primary": true,
"skill_name": "Spark"
},
{
"is_primary": true,
"skill_name": "SQL"
},
{
"is_primary": true,
"skill_name": "Databricks"
},
{
"is_primary": true,
"skill_name": "AWS"
},
{
"is_primary": true,
"skill_name": "Apache NiFi"
},
{
"is_primary": true,
"skill_name": "Lakehouse"
},
{
"is_primary": true,
"skill_name": "Delta Lake"
},
{
"is_primary": true,
"skill_name": "PySpark"
},
{
"is_primary": true,
"skill_name": "SparkSQL"
},
{
"is_primary": true,
"skill_name": "Airflow"
},
{
"is_primary": true,
"skill_name": "Big Data"
},
{
"is_primary": true,
"skill_name": "Data Warehousing"
},
{
"is_primary": true,
"skill_name": "CI/CD"
},
{
"is_primary": false,
"skill_name": "Snowflake"
},
{
"is_primary": false,
"skill_name": "Terraform"
},
{
"is_primary": false,
"skill_name": "Large Language Models"
},
{
"is_primary": false,
"skill_name": "AWS Glue"
},
{
"is_primary": false,
"skill_name": "Amazon S3"
},
{
"is_primary": false,
"skill_name": "Amazon SQS"
},
{
"is_primary": false,
"skill_name": "Amazon SNS"
},
{
"is_primary": false,
"skill_name": "AWS Lambda"
}
],
"jd_role": {
"display_name": "Software Engineer II",
"rationale": null,
"role_aliases": [
"Software Engineer",
"SWE",
"Software Developer"
],
"role_archetype": "Engineering",
"slug": ""
},
"nano_parsed": {
"JD_type": "pass",
"about_company": null,
"certifications": [
"AWS Solutions Architect Associate",
"AWS Developer Associate",
"AWS Data Analytics Specialty",
"Databricks certification"
],
"company_name": "JPMorgan Chase",
"ctc": null,
"domain": {
"primary": {
"aliases": [
"FinTech",
"Banking"
],
"domain": "Financial Services"
},
"secondary": null
},
"education": [],
"experience": {
"max": null,
"min": 2,
"raw": "2+ years applied experience"
},
"job_locations": [],
"role": "Software Engineer II",
"role_aliases": [
"Software Engineer",
"SWE",
"Software Developer"
],
"role_archetype": "Engineering",
"roles_and_responsibilities": [
{
"bullet_count": 6,
"heading": "Job Responsibilities",
"heading_was_present": true,
"source_marker": {
"first_5_words": "\u2022 Execute software solutions, design,",
"last_5_words": "new and emerging technologies."
},
"text": "\u2022 Execute software solutions, design, development, and technical troubleshooting to solve complex problems.\n\u2022 Create secure, high-quality production code and maintain algorithms that run synchronously with systems.\n\u2022 Produce architecture and design artifacts for complex applications, ensuring design constraints are met.\n\u2022 Gather, analyze, and develop visualizations and reporting from large, diverse data sets to improve applications and systems.\n\u2022 Identify hidden problems and patterns in data to drive improvements in coding hygiene and system architecture.\n\u2022 Contribute to software engineering communities of practice and events exploring new and emerging technologies.",
"word_count": 83
},
{
"bullet_count": 7,
"heading": "Required Qualifications, Capabilities, And Skills",
"heading_was_present": true,
"source_marker": {
"first_5_words": "\u2022 Formal training or certification",
"last_5_words": "and CI/CD processes."
},
"text": "\u2022 Formal training or certification on software engineering concepts and 2+ years applied experience\n\u2022 At least one year of practical experience with Spark, SQL, Databricks, and AWS cloud ecosystem.\n\u2022 Expertise in Apache NiFi, Lakehouse or Delta Lake architectures, system design, application development, testing, and operational stability.\n\u2022 Strong programming skills in PySpark and SparkSQL.\n\u2022 Proficient in orchestration using Airflow.\n\u2022 In-depth knowledge of Big Data and data warehousing concepts.\n\u2022 Experience with CI/CD processes.",
"word_count": 85
},
{
"bullet_count": 3,
"heading": "Preferred Qualifications, Capabilities, And Skills",
"heading_was_present": true,
"source_marker": {
"first_5_words": "\u2022 Familiarity with Snowflake, Terraform,",
"last_5_words": "or Databricks certification."
},
"text": "\u2022 Familiarity with Snowflake, Terraform, and large language models.\n\u2022 Exposure to cloud technologies such as AWS Glue, S3, SQS, SNS, Lambda.\n\u2022 AWS certifications such as Solutions Architect Associate, Developer Associate, Data Analytics Specialty, or Databricks certification.",
"word_count": 47
}
],
"urls": []
},
"rejected": false,
"rejection_reason": null,
"run_id": "216829da-da8a-4849-996b-1fd470c98354",
"stage3_signals": {
"alias_found": true,
"alias_match_roles": [
{
"display_name": "Backend Developer",
"kra_matches": null,
"matched_count": null,
"matched_skills": null,
"role_id": 1,
"score": 1.0,
"slug": "backend-engineer",
"total_count": null
}
],
"kra_match_roles": [
{
"display_name": "Data Engineer",
"kra_matches": [
{
"kra_text": "Designs dimensional models, star schemas, data vault structures, and curated data mart tables to support BI tools and self-service analytics consumption.",
"sentence": "Gather, analyze, and develop visualizations and reporting from large, diverse data sets to improve applications and systems.",
"similarity": 0.5819
},
{
"kra_text": "Designs dimensional models, star schemas, data vault structures, and curated data mart tables to support BI tools and self-service analytics consumption.",
"sentence": "In-depth knowledge of Big Data and data warehousing concepts.",
"similarity": 0.5182
},
{
"kra_text": "Develops batch and real-time streaming data pipelines using Apache Spark, Apache Kafka, Apache Flink, or Airflow for data movement and processing at scale.",
"sentence": "At least one year of practical experience with Spark, SQL, Databricks, and AWS cloud ecosystem.",
"similarity": 0.5018
}
],
"matched_count": null,
"matched_skills": null,
"role_id": 2,
"score": 0.534,
"slug": "data-engineer",
"total_count": null
},
{
"display_name": "React Native Developer",
"kra_matches": [
{
"kra_text": "maintain code quality",
"sentence": "Create secure, high-quality production code and maintain algorithms that run synchronously with systems.",
"similarity": 0.5578
},
{
"kra_text": "maintain code quality",
"sentence": "Identify hidden problems and patterns in data to drive improvements in coding hygiene and system architecture.",
"similarity": 0.5237
},
{
"kra_text": "maintain code quality",
"sentence": "Produce architecture and design artifacts for complex applications, ensuring design constraints are met.",
"similarity": 0.4077
}
],
"matched_count": null,
"matched_skills": null,
"role_id": 73,
"score": 0.4964,
"slug": "react-native-developer",
"total_count": null
},
{
"display_name": "Fullstack Developer",
"kra_matches": [
{
"kra_text": "Works closely with product managers and UX designers to translate requirements and wireframes into working software features through iterative development.",
"sentence": "Produce architecture and design artifacts for complex applications, ensuring design constraints are met.",
"similarity": 0.5107
},
{
"kra_text": "Debugs full-stack issues that span frontend rendering, API behavior, database queries, and infrastructure configuration to identify root causes.",
"sentence": "Identify hidden problems and patterns in data to drive improvements in coding hygiene and system architecture.",
"similarity": 0.4996
},
{
"kra_text": "Works closely with product managers and UX designers to translate requirements and wireframes into working software features through iterative development.",
"sentence": "Execute software solutions, design, development, and technical troubleshooting to solve complex problems.",
"similarity": 0.4743
}
],
"matched_count": null,
"matched_skills": null,
"role_id": 15,
"score": 0.4948,
"slug": "full-stack-engineer",
"total_count": null
},
{
"display_name": "Cloud Architect",
"kra_matches": [
{
"kra_text": "Conducts architecture reviews, approves technical design documents, and guides engineering teams through cloud migration and modernization projects.",
"sentence": "Produce architecture and design artifacts for complex applications, ensuring design constraints are met.",
"similarity": 0.5435
},
{
"kra_text": "Conducts architecture reviews, approves technical design documents, and guides engineering teams through cloud migration and modernization projects.",
"sentence": "Execute software solutions, design, development, and technical troubleshooting to solve complex problems.",
"similarity": 0.4761
},
{
"kra_text": "Conducts architecture reviews, approves technical design documents, and guides engineering teams through cloud migration and modernization projects.",
"sentence": "Contribute to software engineering communities of practice and events exploring new and emerging technologies.",
"similarity": 0.4169
}
],
"matched_count": null,
"matched_skills": null,
"role_id": 9,
"score": 0.4788,
"slug": "cloud-architect",
"total_count": null
},
{
"display_name": "Cyber Security Engineer",
"kra_matches": [
{
"kra_text": "Designs and implements security controls including SIEM integration, endpoint detection and response, identity management, and firewall rule management.",
"sentence": "Execute software solutions, design, development, and technical troubleshooting to solve complex problems.",
"similarity": 0.4864
},
{
"kra_text": "Performs threat modeling, security architecture reviews, and quantitative risk analysis for new product features and infrastructure changes.",
"sentence": "Produce architecture and design artifacts for complex applications, ensuring design constraints are met.",
"similarity": 0.4774
},
{
"kra_text": "Defines secure engineering standards, secure coding guidelines, threat intelligence feeds, and compliance requirements for the organization.",
"sentence": "Create secure, high-quality production code and maintain algorithms that run synchronously with systems.",
"similarity": 0.4586
}
],
"matched_count": null,
"matched_skills": null,
"role_id": 5,
"score": 0.4742,
"slug": "cybersecurity-engineer",
"total_count": null
}
],
"skill_match_roles": [
{
"display_name": "ML Engineer",
"kra_matches": null,
"matched_count": 4,
"matched_skills": [
"AWS",
"Airflow",
"CI/CD",
"Delta Lake"
],
"role_id": 3,
"score": 0.3077,
"slug": "ml-engineer",
"total_count": 13
},
{
"display_name": "MLOps Engineer",
"kra_matches": null,
"matched_count": 3,
"matched_skills": [
"AWS",
"Airflow",
"Delta Lake"
],
"role_id": 16,
"score": 0.2308,
"slug": "ml-ops-engineer",
"total_count": 13
},
{
"display_name": "Data Engineer",
"kra_matches": null,
"matched_count": 3,
"matched_skills": [
"AWS",
"Apache Spark",
"SQL"
],
"role_id": 2,
"score": 0.2308,
"slug": "data-engineer",
"total_count": 13
},
{
"display_name": "DevOps Engineer",
"kra_matches": null,
"matched_count": 2,
"matched_skills": [
"AWS",
"CI/CD"
],
"role_id": 10,
"score": 0.1538,
"slug": "devops-engineer",
"total_count": 13
},
{
"display_name": "Engineering Manager",
"kra_matches": null,
"matched_count": 2,
"matched_skills": [
"AWS",
"SQL"
],
"role_id": 121,
"score": 0.1538,
"slug": "engineering-manager",
"total_count": 13
}
]
},
"stage4_decision": {
"alias_collision_detected": false,
"case": "DOMAIN",
"chosen_role": {
"display_name": "Data Engineer",
"kra_matches": null,
"matched_count": null,
"matched_skills": null,
"role_id": 2,
"score": 0.97,
"slug": "data-engineer",
"total_count": null
},
"confidence": 0.97,
"is_new_role": false,
"llm2_fired": false,
"llm2_reasoning": null,
"matched_dimensions": [
"Big Data Engineering",
"Data Pipeline Orchestration",
"Lakehouse and Data Warehousing",
"Cloud Data Engineering",
"Software Design and Architecture",
"Production Troubleshooting and Stability"
],
"matched_kras": [
"Execute software solutions, design, development, and technical troubleshooting",
"Create secure, high-quality production code",
"Produce architecture and design artifacts for complex applications",
"Gather, analyze, and develop visualizations and reporting",
"Identify hidden problems and patterns in data",
"Contribute to software engineering communities of practice"
],
"matched_skills": [
"Spark",
"SQL",
"Databricks",
"AWS",
"Apache NiFi",
"Lakehouse",
"Delta Lake",
"PySpark",
"SparkSQL",
"Airflow",
"Snowflake",
"Terraform",
"AWS Glue",
"S3",
"SQS",
"SNS",
"Lambda",
"CI/CD"
],
"new_role_display_name": null,
"new_role_slug": null,
"queued": false,
"reasoning": "Domain=Data Engineering \u0026 Analytics; The JD is centered on Spark, Databricks, AWS, Airflow, big data, and lakehouse/data warehousing work, which most closely matches Data Engineer.",
"sub_role": null
},
"stage5_updates": {
"centroid_n_after": 384,
"centroid_updated": true,
"collision_log_id": null,
"new_kra_attached": null,
"new_skills_attached": [
{
"is_primary": true,
"queue_id": 18209,
"role_display_name": "Data Engineer",
"role_slug": "data-engineer",
"skill_name": "Apache NiFi",
"status": "pending"
},
{
"is_primary": true,
"queue_id": 18210,
"role_display_name": "Data Engineer",
"role_slug": "data-engineer",
"skill_name": "PySpark",
"status": "pending"
},
{
"is_primary": true,
"queue_id": 18211,
"role_display_name": "Data Engineer",
"role_slug": "data-engineer",
"skill_name": "SparkSQL",
"status": "pending"
},
{
"is_primary": true,
"queue_id": 18212,
"role_display_name": "Data Engineer",
"role_slug": "data-engineer",
"skill_name": "Big Data",
"status": "pending"
},
{
"is_primary": true,
"queue_id": 18213,
"role_display_name": "Data Engineer",
"role_slug": "data-engineer",
"skill_name": "Data Warehousing",
"status": "pending"
},
{
"is_primary": false,
"queue_id": 18214,
"role_display_name": "Data Engineer",
"role_slug": "data-engineer",
"skill_name": "Large Language Models",
"status": "pending"
},
{
"is_primary": false,
"queue_id": 18215,
"role_display_name": "Data Engineer",
"role_slug": "data-engineer",
"skill_name": "AWS Glue",
"status": "pending"
},
{
"is_primary": false,
"queue_id": 18216,
"role_display_name": "Data Engineer",
"role_slug": "data-engineer",
"skill_name": "Amazon SNS",
"status": "pending"
}
],
"queue_entry_id": null,
"v3_pipeline_triggered": false,
"v3_role_slug": null,
"v3_run_id": null
}
}
API 2 — extract-details
{
"alias_matches": [
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 2510,
"existing_alias_text": "spark",
"input_term": "Spark",
"matched_canonical": {
"category_id": 5,
"display_name": "Apache Spark",
"id": 1350,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "FRAMEWORK",
"slug": "apache-spark",
"sub_category_id": 1021,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "alias"
},
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 271,
"existing_alias_text": "SQL",
"input_term": "SQL",
"matched_canonical": {
"category_id": 6,
"display_name": "SQL",
"id": 101,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "LANGUAGE",
"slug": "sql",
"sub_category_id": 97,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "alias"
},
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 1838,
"existing_alias_text": "Databricks",
"input_term": "Databricks",
"matched_canonical": {
"category_id": 9,
"display_name": "Databricks",
"id": 1202,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "PLATFORM",
"slug": "databricks",
"sub_category_id": 911,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "alias"
},
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 406,
"existing_alias_text": "AWS",
"input_term": "AWS",
"matched_canonical": {
"category_id": 9,
"display_name": "AWS",
"id": 187,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "PLATFORM",
"slug": "aws",
"sub_category_id": 46,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "alias"
},
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 2018,
"existing_alias_text": "Lakehouse",
"input_term": "Lakehouse",
"matched_canonical": {
"category_id": 1,
"display_name": "Lakehouse",
"id": 1359,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "PATTERN",
"slug": "lakehouse",
"sub_category_id": 1026,
"typical_lifespan": "EVERGREEN",
"volatility": "EMERGING"
},
"matched_via": "alias"
},
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 498,
"existing_alias_text": "Delta Lake",
"input_term": "Delta Lake",
"matched_canonical": {
"category_id": 13,
"display_name": "Delta Lake",
"id": 237,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "TOOL",
"slug": "delta-lake",
"sub_category_id": 1170,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "alias"
},
{
"alias_persist_skipped_reason": "TODO: REMOVE AFTER TESTING \u2014 alias DB write disabled",
"alias_persisted": false,
"existing_alias_id": 2004,
"existing_alias_text": "Apache Spark",
"input_term": "PySpark",
"matched_canonical": {
"category_id": 5,
"display_name": "Apache Spark",
"id": 1350,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "FRAMEWORK",
"slug": "apache-spark",
"sub_category_id": 1021,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "embedding_alias"
},
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 526,
"existing_alias_text": "Airflow",
"input_term": "Airflow",
"matched_canonical": {
"category_id": 13,
"display_name": "Airflow",
"id": 265,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "TOOL",
"slug": "airflow",
"sub_category_id": 130,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "alias"
},
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 1826,
"existing_alias_text": "CI/CD",
"input_term": "CI/CD",
"matched_canonical": {
"category_id": 8,
"display_name": "CI/CD",
"id": 1190,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "METHODOLOGY",
"slug": "ci-cd",
"sub_category_id": 900,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "alias"
},
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 299,
"existing_alias_text": "Snowflake",
"input_term": "Snowflake",
"matched_canonical": {
"category_id": 9,
"display_name": "Snowflake",
"id": 105,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "PLATFORM",
"slug": "snowflake",
"sub_category_id": 113,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "alias"
},
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 547,
"existing_alias_text": "Terraform",
"input_term": "Terraform",
"matched_canonical": {
"category_id": 13,
"display_name": "Terraform",
"id": 286,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "TOOL",
"slug": "terraform",
"sub_category_id": 191,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "alias"
},
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 379,
"existing_alias_text": "Amazon S3",
"input_term": "Amazon S3",
"matched_canonical": {
"category_id": 11,
"display_name": "Amazon S3",
"id": 170,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "CLOUD_SERVICE",
"slug": "amazon-s3",
"sub_category_id": 120,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "alias"
},
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 176,
"existing_alias_text": "Amazon SQS",
"input_term": "Amazon SQS",
"matched_canonical": {
"category_id": 11,
"display_name": "Amazon SQS",
"id": 39,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "CLOUD_SERVICE",
"slug": "amazon-sqs",
"sub_category_id": 56,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "alias"
},
{
"alias_persist_skipped_reason": "TODO: REMOVE AFTER TESTING \u2014 alias DB write disabled",
"alias_persisted": false,
"existing_alias_id": 5151,
"existing_alias_text": "AWS SNS",
"input_term": "Amazon SNS",
"matched_canonical": {
"category_id": 11,
"display_name": "AWS SNS",
"id": 3579,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "CLOUD_SERVICE",
"slug": "aws-sns",
"sub_category_id": 2847,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "embedding_alias"
},
{
"alias_persist_skipped_reason": "alias_text already exists for this canonical skill",
"alias_persisted": false,
"existing_alias_id": 3563,
"existing_alias_text": "AWS Lambda",
"input_term": "AWS Lambda",
"matched_canonical": {
"category_id": 11,
"display_name": "AWS Lambda",
"id": 2295,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "CLOUD_SERVICE",
"slug": "aws-lambda",
"sub_category_id": 1704,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"matched_via": "alias"
}
],
"candidate_roles": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
},
{
"display_name": "Pega Developer",
"id": 24,
"rationale": null,
"role_archetype": null,
"slug": "pega-developer",
"source": "db"
},
{
"display_name": "Engineering Manager",
"id": 121,
"rationale": null,
"role_archetype": null,
"slug": "engineering-manager",
"source": "db"
},
{
"display_name": ".NET Backend Developer",
"id": 83,
"rationale": null,
"role_archetype": "Engineering",
"slug": "dotnet-backend-developer",
"source": "db"
},
{
"display_name": "Backend Developer",
"id": 1,
"rationale": null,
"role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
"slug": "backend-engineer",
"source": "db"
},
{
"display_name": "Cyber Security Engineer",
"id": 5,
"rationale": null,
"role_archetype": null,
"slug": "cybersecurity-engineer",
"source": "db"
},
{
"display_name": "DevOps Engineer",
"id": 10,
"rationale": null,
"role_archetype": null,
"slug": "devops-engineer",
"source": "db"
},
{
"display_name": "Fullstack Developer",
"id": 15,
"rationale": null,
"role_archetype": null,
"slug": "full-stack-engineer",
"source": "db"
},
{
"display_name": "Go Backend Developer",
"id": 81,
"rationale": null,
"role_archetype": "Engineering",
"slug": "go-backend-developer",
"source": "db"
},
{
"display_name": "Java Backend Developer",
"id": 79,
"rationale": null,
"role_archetype": "Engineering",
"slug": "java-backend-developer",
"source": "db"
},
{
"display_name": "Kotlin Backend Developer",
"id": 84,
"rationale": null,
"role_archetype": "Engineering",
"slug": "kotlin-server-backend-developer",
"source": "db"
},
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
},
{
"display_name": "Node.js Backend Developer",
"id": 82,
"rationale": null,
"role_archetype": "Engineering",
"slug": "node-backend-developer",
"source": "db"
},
{
"display_name": "Python Backend Developer",
"id": 80,
"rationale": null,
"role_archetype": "Engineering",
"slug": "python-backend-developer",
"source": "db"
},
{
"display_name": "Scala Backend Developer",
"id": 87,
"rationale": null,
"role_archetype": "Engineering",
"slug": "scala-backend-developer",
"source": "db"
},
{
"display_name": "AI Engineer",
"id": 13,
"rationale": null,
"role_archetype": null,
"slug": "ai-engineer",
"source": "db"
},
{
"display_name": "Cloud Architect",
"id": 9,
"rationale": null,
"role_archetype": null,
"slug": "cloud-architect",
"source": "db"
},
{
"display_name": "Cloud Security Engineer",
"id": 23,
"rationale": null,
"role_archetype": null,
"slug": "cloud-security-engineer",
"source": "db"
},
{
"display_name": "PHP Backend Developer",
"id": 86,
"rationale": null,
"role_archetype": "Engineering",
"slug": "php-backend-developer",
"source": "db"
},
{
"display_name": "Ruby Backend Developer",
"id": 85,
"rationale": null,
"role_archetype": "Engineering",
"slug": "ruby-backend-developer",
"source": "db"
},
{
"display_name": "Web Developer",
"id": 25,
"rationale": null,
"role_archetype": null,
"slug": "web-developer",
"source": "db"
}
],
"chosen_role": {
"display_name": "Data Engineer",
"id": 2,
"rationale": "Domain=Data Engineering \u0026 Analytics; The JD is centered on Spark, Databricks, AWS, Airflow, big data, and lakehouse/data warehousing work, which most closely matches Data Engineer.",
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "ETL and ELT Tooling",
"id": 24,
"rationale": "Packaged tools for extracting, loading, and transforming data across systems. This dimension covers connector-based ingestion, transformation frameworks, and managed integration products.",
"slug": "etl-and-elt-tooling",
"source": "db"
},
"input_skill": "Spark",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Pega Programming Languages \u0026 DSLs",
"id": 267,
"rationale": "Programming languages and domain-specific languages used in Pega development.",
"slug": "pega-programming-languages-dsls",
"source": "db"
},
"input_skill": "SQL",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Pega Developer",
"id": 24,
"rationale": null,
"role_archetype": null,
"slug": "pega-developer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages \u0026 DSLs",
"id": 475,
"rationale": "Oversee and guide the selection and effective use of programming and domain\u2010specific languages in software projects.",
"slug": "programming-languages-dsls",
"source": "db"
},
"input_skill": "SQL",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Engineering Manager",
"id": 121,
"rationale": null,
"role_archetype": null,
"slug": "engineering-manager",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages for Data Work",
"id": 21,
"rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
"slug": "programming-languages-for-data-work",
"source": "db"
},
"input_skill": "SQL",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "React Frontend Development",
"id": 96,
"rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
"slug": "d_init_01",
"source": "db"
},
"input_skill": "Databricks",
"llm_role": null,
"roles_from_db": []
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Platforms",
"id": 20,
"rationale": "Underlying cloud providers that host the managed services or infrastructure used by the role, such as AWS, Azure, and GCP.",
"slug": "cloud-platforms",
"source": "db"
},
"input_skill": "AWS",
"llm_role": null,
"roles_from_db": [
{
"display_name": ".NET Backend Developer",
"id": 83,
"rationale": null,
"role_archetype": "Engineering",
"slug": "dotnet-backend-developer",
"source": "db"
},
{
"display_name": "Backend Developer",
"id": 1,
"rationale": null,
"role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
"slug": "backend-engineer",
"source": "db"
},
{
"display_name": "Cyber Security Engineer",
"id": 5,
"rationale": null,
"role_archetype": null,
"slug": "cybersecurity-engineer",
"source": "db"
},
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
},
{
"display_name": "DevOps Engineer",
"id": 10,
"rationale": null,
"role_archetype": null,
"slug": "devops-engineer",
"source": "db"
},
{
"display_name": "Fullstack Developer",
"id": 15,
"rationale": null,
"role_archetype": null,
"slug": "full-stack-engineer",
"source": "db"
},
{
"display_name": "Go Backend Developer",
"id": 81,
"rationale": null,
"role_archetype": "Engineering",
"slug": "go-backend-developer",
"source": "db"
},
{
"display_name": "Java Backend Developer",
"id": 79,
"rationale": null,
"role_archetype": "Engineering",
"slug": "java-backend-developer",
"source": "db"
},
{
"display_name": "Kotlin Backend Developer",
"id": 84,
"rationale": null,
"role_archetype": "Engineering",
"slug": "kotlin-server-backend-developer",
"source": "db"
},
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
},
{
"display_name": "Node.js Backend Developer",
"id": 82,
"rationale": null,
"role_archetype": "Engineering",
"slug": "node-backend-developer",
"source": "db"
},
{
"display_name": "Python Backend Developer",
"id": 80,
"rationale": null,
"role_archetype": "Engineering",
"slug": "python-backend-developer",
"source": "db"
},
{
"display_name": "Scala Backend Developer",
"id": 87,
"rationale": null,
"role_archetype": "Engineering",
"slug": "scala-backend-developer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Platforms for AI Deployment",
"id": 211,
"rationale": "Major cloud services that provide infrastructure and managed services for AI workloads.",
"slug": "cloud-platforms-for-ai-deployment",
"source": "db"
},
"input_skill": "AWS",
"llm_role": null,
"roles_from_db": [
{
"display_name": "AI Engineer",
"id": 13,
"rationale": null,
"role_archetype": null,
"slug": "ai-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Provider Platforms",
"id": 131,
"rationale": "Major cloud platforms and their core service ecosystems used to design target-state architectures, choose deployment boundaries, and evaluate managed capabilities. This is the primary substrate for cloud architecture decisions.",
"slug": "cloud-provider-platforms",
"source": "db"
},
"input_skill": "AWS",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Cloud Architect",
"id": 9,
"rationale": null,
"role_archetype": null,
"slug": "cloud-architect",
"source": "db"
},
{
"display_name": "Cloud Security Engineer",
"id": 23,
"rationale": null,
"role_archetype": null,
"slug": "cloud-security-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Security Posture Tools",
"id": 64,
"rationale": "Cloud-native security platforms used to assess misconfiguration, workload exposure, and cloud control coverage. This dimension includes the major CNAPP/CSPM/CWPP vendors and cloud security services the role reviews and tunes.",
"slug": "cloud-security-posture-tools",
"source": "db"
},
"input_skill": "AWS",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Cloud Security Engineer",
"id": 23,
"rationale": null,
"role_archetype": null,
"slug": "cloud-security-engineer",
"source": "db"
},
{
"display_name": "Cyber Security Engineer",
"id": 5,
"rationale": null,
"role_archetype": null,
"slug": "cybersecurity-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Vendor Product Families",
"id": 477,
"rationale": "Coordinate usage, licensing, and architecture decisions for major vendor software and cloud product families.",
"slug": "vendor-product-families",
"source": "db"
},
"input_skill": "AWS",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Engineering Manager",
"id": 121,
"rationale": null,
"role_archetype": null,
"slug": "engineering-manager",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "React Frontend Development",
"id": 96,
"rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
"slug": "d_init_01",
"source": "db"
},
"input_skill": "Lakehouse",
"llm_role": null,
"roles_from_db": []
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Model and Data Versioning",
"id": 48,
"rationale": "Versioning systems for datasets, features, and model artifacts at the storage layer. This enables reproducible training, rollback, lineage of artifacts, and controlled promotion of model assets.",
"slug": "model-and-data-versioning",
"source": "db"
},
"input_skill": "Delta Lake",
"llm_role": null,
"roles_from_db": [
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "ETL and ELT Tooling",
"id": 24,
"rationale": "Packaged tools for extracting, loading, and transforming data across systems. This dimension covers connector-based ingestion, transformation frameworks, and managed integration products.",
"slug": "etl-and-elt-tooling",
"source": "db"
},
"input_skill": "PySpark",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Workflow Orchestration for ML Pipelines",
"id": 54,
"rationale": "Workflow engines used to coordinate training, evaluation, deployment, and retraining jobs. This cluster covers dependencies, retries, scheduling, and pipeline composition for ML lifecycle automation.",
"slug": "workflow-orchestration-for-ml-pipelines",
"source": "db"
},
"input_skill": "Airflow",
"llm_role": null,
"roles_from_db": [
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "CI/CD Pipeline Platforms",
"id": 150,
"rationale": "Systems used to define, run, and maintain automated build and deployment workflows. This cluster is coherent because the role owns delivery automation end to end, including pipeline reliability and promotion logic.",
"slug": "ci-cd-pipeline-platforms",
"source": "db"
},
"input_skill": "CI/CD",
"llm_role": null,
"roles_from_db": [
{
"display_name": "DevOps Engineer",
"id": 10,
"rationale": null,
"role_archetype": null,
"slug": "devops-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "CI/CD for Machine Learning",
"id": 56,
"rationale": "Tools and platforms for automating ML model integration, testing, and deployment pipelines.",
"slug": "ci-cd-for-machine-learning",
"source": "db"
},
"input_skill": "CI/CD",
"llm_role": null,
"roles_from_db": [
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Data Warehouses",
"id": 22,
"rationale": "Managed analytical storage and compute platforms used for curated datasets, reporting, and downstream analytics. These systems are central to data modeling, performance tuning, and cost-aware query design.",
"slug": "cloud-data-warehouses",
"source": "db"
},
"input_skill": "Snowflake",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Infrastructure \u0026 Security Automation Frameworks",
"id": 249,
"rationale": "Frameworks and libraries for provisioning, configuring, and automating cloud security infrastructure.",
"slug": "infrastructure-security-automation-frameworks",
"source": "db"
},
"input_skill": "Terraform",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Cloud Security Engineer",
"id": 23,
"rationale": null,
"role_archetype": null,
"slug": "cloud-security-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Infrastructure as Code",
"id": 132,
"rationale": "Declarative provisioning and environment definition tools used to codify cloud infrastructure, repeatable environments, and platform standards. Cloud Architects use these to express reference architectures and guardrails.",
"slug": "infrastructure-as-code",
"source": "db"
},
"input_skill": "Terraform",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Cloud Architect",
"id": 9,
"rationale": null,
"role_archetype": null,
"slug": "cloud-architect",
"source": "db"
},
{
"display_name": "DevOps Engineer",
"id": 10,
"rationale": null,
"role_archetype": null,
"slug": "devops-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Infrastructure as Code for ML",
"id": 57,
"rationale": "Tools for provisioning and managing ML infrastructure resources through code.",
"slug": "infrastructure-as-code-for-ml",
"source": "db"
},
"input_skill": "Terraform",
"llm_role": null,
"roles_from_db": [
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Storage and Data Services",
"id": 144,
"rationale": "Cloud-native storage and managed data services used to place workloads, choose durability tiers, and define platform boundaries. This is a coherent cluster because architects evaluate storage fit, access patterns, and managed service tradeoffs.",
"slug": "cloud-storage-and-data-services",
"source": "db"
},
"input_skill": "Amazon S3",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Cloud Architect",
"id": 9,
"rationale": null,
"role_archetype": null,
"slug": "cloud-architect",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Storage and File Formats",
"id": 35,
"rationale": "Object storage and data file formats used as the physical substrate for data movement and lake-style analytics. Data engineers need these to manage landing zones, partitioned datasets, and efficient interchange.",
"slug": "cloud-storage-and-file-formats",
"source": "db"
},
"input_skill": "Amazon S3",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Messaging and Background Jobs",
"id": 291,
"rationale": "Asynchronous processing patterns and worker systems used to decouple backend work from request handling. This is a coherent cluster because the role supports background jobs, retries, and deferred processing.",
"slug": "messaging-and-background-jobs",
"source": "db"
},
"input_skill": "Amazon SQS",
"llm_role": null,
"roles_from_db": [
{
"display_name": "PHP Backend Developer",
"id": 86,
"rationale": null,
"role_archetype": "Engineering",
"slug": "php-backend-developer",
"source": "db"
},
{
"display_name": "Python Backend Developer",
"id": 80,
"rationale": null,
"role_archetype": "Engineering",
"slug": "python-backend-developer",
"source": "db"
},
{
"display_name": "Ruby Backend Developer",
"id": 85,
"rationale": null,
"role_archetype": "Engineering",
"slug": "ruby-backend-developer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Messaging and Event Streaming",
"id": 8,
"rationale": "Transport-layer systems used to move events and decouple producers from consumers. Data engineers use these systems to ingest, buffer, and distribute event data before downstream processing.",
"slug": "messaging-and-event-streaming",
"source": "db"
},
"input_skill": "Amazon SQS",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Backend Developer",
"id": 1,
"rationale": null,
"role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
"slug": "backend-engineer",
"source": "db"
},
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Asynchronous Messaging and Event Streaming",
"id": 297,
"rationale": "Asynchronous communication patterns and broker technologies used to decouple backend services and move work off the request path. Includes queues, pub/sub, event streams, consumer groups, dead-letter queues, and delivery semantics across systems such as Kafka, RabbitMQ, NATS, SQS/SNS, Pulsar, and ActiveMQ.",
"slug": "asynchronous-messaging-and-event-streaming",
"source": "db"
},
"input_skill": "Amazon SNS",
"llm_role": null,
"roles_from_db": [
{
"display_name": ".NET Backend Developer",
"id": 83,
"rationale": null,
"role_archetype": "Engineering",
"slug": "dotnet-backend-developer",
"source": "db"
},
{
"display_name": "Go Backend Developer",
"id": 81,
"rationale": null,
"role_archetype": "Engineering",
"slug": "go-backend-developer",
"source": "db"
},
{
"display_name": "Kotlin Backend Developer",
"id": 84,
"rationale": null,
"role_archetype": "Engineering",
"slug": "kotlin-server-backend-developer",
"source": "db"
},
{
"display_name": "Node.js Backend Developer",
"id": 82,
"rationale": null,
"role_archetype": "Engineering",
"slug": "node-backend-developer",
"source": "db"
},
{
"display_name": "Scala Backend Developer",
"id": 87,
"rationale": null,
"role_archetype": "Engineering",
"slug": "scala-backend-developer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Platforms",
"id": 20,
"rationale": "Underlying cloud providers that host the managed services or infrastructure used by the role, such as AWS, Azure, and GCP.",
"slug": "cloud-platforms",
"source": "db"
},
"input_skill": "AWS Lambda",
"llm_role": null,
"roles_from_db": [
{
"display_name": ".NET Backend Developer",
"id": 83,
"rationale": null,
"role_archetype": "Engineering",
"slug": "dotnet-backend-developer",
"source": "db"
},
{
"display_name": "Backend Developer",
"id": 1,
"rationale": null,
"role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
"slug": "backend-engineer",
"source": "db"
},
{
"display_name": "Cyber Security Engineer",
"id": 5,
"rationale": null,
"role_archetype": null,
"slug": "cybersecurity-engineer",
"source": "db"
},
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
},
{
"display_name": "DevOps Engineer",
"id": 10,
"rationale": null,
"role_archetype": null,
"slug": "devops-engineer",
"source": "db"
},
{
"display_name": "Fullstack Developer",
"id": 15,
"rationale": null,
"role_archetype": null,
"slug": "full-stack-engineer",
"source": "db"
},
{
"display_name": "Go Backend Developer",
"id": 81,
"rationale": null,
"role_archetype": "Engineering",
"slug": "go-backend-developer",
"source": "db"
},
{
"display_name": "Java Backend Developer",
"id": 79,
"rationale": null,
"role_archetype": "Engineering",
"slug": "java-backend-developer",
"source": "db"
},
{
"display_name": "Kotlin Backend Developer",
"id": 84,
"rationale": null,
"role_archetype": "Engineering",
"slug": "kotlin-server-backend-developer",
"source": "db"
},
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
},
{
"display_name": "Node.js Backend Developer",
"id": 82,
"rationale": null,
"role_archetype": "Engineering",
"slug": "node-backend-developer",
"source": "db"
},
{
"display_name": "Python Backend Developer",
"id": 80,
"rationale": null,
"role_archetype": "Engineering",
"slug": "python-backend-developer",
"source": "db"
},
{
"display_name": "Scala Backend Developer",
"id": 87,
"rationale": null,
"role_archetype": "Engineering",
"slug": "scala-backend-developer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Platforms \u0026 Hosting Providers",
"id": 278,
"rationale": "Familiarity with vendor-specific hosting and backend services for deploying and scaling web applications.",
"slug": "cloud-platforms-hosting-providers",
"source": "db"
},
"input_skill": "AWS Lambda",
"llm_role": null,
"roles_from_db": [
{
"display_name": ".NET Backend Developer",
"id": 83,
"rationale": null,
"role_archetype": "Engineering",
"slug": "dotnet-backend-developer",
"source": "db"
},
{
"display_name": "Kotlin Backend Developer",
"id": 84,
"rationale": null,
"role_archetype": "Engineering",
"slug": "kotlin-server-backend-developer",
"source": "db"
},
{
"display_name": "Scala Backend Developer",
"id": 87,
"rationale": null,
"role_archetype": "Engineering",
"slug": "scala-backend-developer",
"source": "db"
},
{
"display_name": "Web Developer",
"id": 25,
"rationale": null,
"role_archetype": null,
"slug": "web-developer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Platforms \u0026 Managed Services",
"id": 221,
"rationale": "Operates and integrates vendor-specific cloud compute, storage, and hosting services.",
"slug": "cloud-platforms-managed-services",
"source": "db"
},
"input_skill": "AWS Lambda",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Fullstack Developer",
"id": 15,
"rationale": null,
"role_archetype": null,
"slug": "full-stack-engineer",
"source": "db"
},
{
"display_name": "Go Backend Developer",
"id": 81,
"rationale": null,
"role_archetype": "Engineering",
"slug": "go-backend-developer",
"source": "db"
},
{
"display_name": "Node.js Backend Developer",
"id": 82,
"rationale": null,
"role_archetype": "Engineering",
"slug": "node-backend-developer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Platforms \u0026 Services",
"id": 296,
"rationale": "Deploying, configuring, and managing Python backends on major cloud and PaaS offerings.",
"slug": "cloud-platforms-services",
"source": "db"
},
"input_skill": "AWS Lambda",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Python Backend Developer",
"id": 80,
"rationale": null,
"role_archetype": "Engineering",
"slug": "python-backend-developer",
"source": "db"
}
]
}
],
"input_final_skills": [
"Spark",
"SQL",
"Databricks",
"AWS",
"Apache NiFi",
"Lakehouse",
"Delta Lake",
"PySpark",
"SparkSQL",
"Airflow",
"Big Data",
"Data Warehousing",
"CI/CD",
"Snowflake",
"Terraform",
"Large Language Models",
"AWS Glue",
"Amazon S3",
"Amazon SQS",
"Amazon SNS",
"AWS Lambda"
],
"input_llm_skills": [
"Spark",
"SQL",
"Databricks",
"AWS",
"Apache NiFi",
"Lakehouse",
"Delta Lake",
"PySpark",
"SparkSQL",
"Airflow",
"Big Data",
"Data Warehousing",
"CI/CD",
"Snowflake",
"Terraform",
"Large Language Models",
"AWS Glue",
"Amazon S3",
"Amazon SQS",
"Amazon SNS",
"AWS Lambda"
],
"new_aliases_persisted": 0,
"run_id": "216829da-da8a-4849-996b-1fd470c98354",
"skills_detail": [
{
"aliases_in_db": [
{
"alias_text": "Apache Spark",
"alias_type": "CANONICAL",
"id": 2004,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "apache spark 3",
"alias_type": "VERSION",
"id": 2006,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "spark",
"alias_type": "VERSION",
"id": 2510,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "spark 3",
"alias_type": "VERSION",
"id": 2007,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "spark 3.x",
"alias_type": "VERSION",
"id": 2009,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "spark3",
"alias_type": "VERSION",
"id": 2008,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 5,
"display_name": "Apache Spark",
"id": 1350,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "FRAMEWORK",
"slug": "apache-spark",
"sub_category_id": 1021,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "ETL and ELT Tooling",
"id": 24,
"rationale": "Packaged tools for extracting, loading, and transforming data across systems. This dimension covers connector-based ingestion, transformation frameworks, and managed integration products.",
"slug": "etl-and-elt-tooling",
"source": "db"
},
"input_skill": "Spark",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
}
],
"input_skill": "Spark",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "SQL",
"alias_type": "CANONICAL",
"id": 271,
"is_primary": true,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 6,
"display_name": "SQL",
"id": 101,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "LANGUAGE",
"slug": "sql",
"sub_category_id": 97,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Pega Programming Languages \u0026 DSLs",
"id": 267,
"rationale": "Programming languages and domain-specific languages used in Pega development.",
"slug": "pega-programming-languages-dsls",
"source": "db"
},
"input_skill": "SQL",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Pega Developer",
"id": 24,
"rationale": null,
"role_archetype": null,
"slug": "pega-developer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages \u0026 DSLs",
"id": 475,
"rationale": "Oversee and guide the selection and effective use of programming and domain\u2010specific languages in software projects.",
"slug": "programming-languages-dsls",
"source": "db"
},
"input_skill": "SQL",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Engineering Manager",
"id": 121,
"rationale": null,
"role_archetype": null,
"slug": "engineering-manager",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages for Data Work",
"id": 21,
"rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
"slug": "programming-languages-for-data-work",
"source": "db"
},
"input_skill": "SQL",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
}
],
"input_skill": "SQL",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "Databricks",
"alias_type": "CANONICAL",
"id": 1838,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 9,
"display_name": "Databricks",
"id": 1202,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "PLATFORM",
"slug": "databricks",
"sub_category_id": 911,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "React Frontend Development",
"id": 96,
"rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
"slug": "d_init_01",
"source": "db"
},
"input_skill": "Databricks",
"llm_role": null,
"roles_from_db": []
}
],
"input_skill": "Databricks",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "AWS",
"alias_type": "CANONICAL",
"id": 406,
"is_primary": true,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 9,
"display_name": "AWS",
"id": 187,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "PLATFORM",
"slug": "aws",
"sub_category_id": 46,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Platforms",
"id": 20,
"rationale": "Underlying cloud providers that host the managed services or infrastructure used by the role, such as AWS, Azure, and GCP.",
"slug": "cloud-platforms",
"source": "db"
},
"input_skill": "AWS",
"llm_role": null,
"roles_from_db": [
{
"display_name": ".NET Backend Developer",
"id": 83,
"rationale": null,
"role_archetype": "Engineering",
"slug": "dotnet-backend-developer",
"source": "db"
},
{
"display_name": "Backend Developer",
"id": 1,
"rationale": null,
"role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
"slug": "backend-engineer",
"source": "db"
},
{
"display_name": "Cyber Security Engineer",
"id": 5,
"rationale": null,
"role_archetype": null,
"slug": "cybersecurity-engineer",
"source": "db"
},
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
},
{
"display_name": "DevOps Engineer",
"id": 10,
"rationale": null,
"role_archetype": null,
"slug": "devops-engineer",
"source": "db"
},
{
"display_name": "Fullstack Developer",
"id": 15,
"rationale": null,
"role_archetype": null,
"slug": "full-stack-engineer",
"source": "db"
},
{
"display_name": "Go Backend Developer",
"id": 81,
"rationale": null,
"role_archetype": "Engineering",
"slug": "go-backend-developer",
"source": "db"
},
{
"display_name": "Java Backend Developer",
"id": 79,
"rationale": null,
"role_archetype": "Engineering",
"slug": "java-backend-developer",
"source": "db"
},
{
"display_name": "Kotlin Backend Developer",
"id": 84,
"rationale": null,
"role_archetype": "Engineering",
"slug": "kotlin-server-backend-developer",
"source": "db"
},
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
},
{
"display_name": "Node.js Backend Developer",
"id": 82,
"rationale": null,
"role_archetype": "Engineering",
"slug": "node-backend-developer",
"source": "db"
},
{
"display_name": "Python Backend Developer",
"id": 80,
"rationale": null,
"role_archetype": "Engineering",
"slug": "python-backend-developer",
"source": "db"
},
{
"display_name": "Scala Backend Developer",
"id": 87,
"rationale": null,
"role_archetype": "Engineering",
"slug": "scala-backend-developer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Platforms for AI Deployment",
"id": 211,
"rationale": "Major cloud services that provide infrastructure and managed services for AI workloads.",
"slug": "cloud-platforms-for-ai-deployment",
"source": "db"
},
"input_skill": "AWS",
"llm_role": null,
"roles_from_db": [
{
"display_name": "AI Engineer",
"id": 13,
"rationale": null,
"role_archetype": null,
"slug": "ai-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Provider Platforms",
"id": 131,
"rationale": "Major cloud platforms and their core service ecosystems used to design target-state architectures, choose deployment boundaries, and evaluate managed capabilities. This is the primary substrate for cloud architecture decisions.",
"slug": "cloud-provider-platforms",
"source": "db"
},
"input_skill": "AWS",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Cloud Architect",
"id": 9,
"rationale": null,
"role_archetype": null,
"slug": "cloud-architect",
"source": "db"
},
{
"display_name": "Cloud Security Engineer",
"id": 23,
"rationale": null,
"role_archetype": null,
"slug": "cloud-security-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Security Posture Tools",
"id": 64,
"rationale": "Cloud-native security platforms used to assess misconfiguration, workload exposure, and cloud control coverage. This dimension includes the major CNAPP/CSPM/CWPP vendors and cloud security services the role reviews and tunes.",
"slug": "cloud-security-posture-tools",
"source": "db"
},
"input_skill": "AWS",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Cloud Security Engineer",
"id": 23,
"rationale": null,
"role_archetype": null,
"slug": "cloud-security-engineer",
"source": "db"
},
{
"display_name": "Cyber Security Engineer",
"id": 5,
"rationale": null,
"role_archetype": null,
"slug": "cybersecurity-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Vendor Product Families",
"id": 477,
"rationale": "Coordinate usage, licensing, and architecture decisions for major vendor software and cloud product families.",
"slug": "vendor-product-families",
"source": "db"
},
"input_skill": "AWS",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Engineering Manager",
"id": 121,
"rationale": null,
"role_archetype": null,
"slug": "engineering-manager",
"source": "db"
}
]
}
],
"input_skill": "AWS",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [],
"canonical": null,
"dimensions": [],
"input_skill": "Apache NiFi",
"matched_via": null,
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": {
"derived": {
"category": "Data Engineering Tools",
"skill_nature": "TOOL",
"sub_category": "general",
"typical_lifespan": "MULTI_YEAR",
"version_strategy": "UNVERSIONED",
"volatility": "MEDIUM"
},
"enrichment": null,
"keep_log": [],
"locked_dimensions": [],
"merge_log": [],
"placed": null,
"relationships": null,
"skill_id": "apache-nifi",
"split_log": [],
"typed": null,
"warnings": []
},
"source_tag": "llm",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "Lakehouse",
"alias_type": "CANONICAL",
"id": 2018,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 1,
"display_name": "Lakehouse",
"id": 1359,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "PATTERN",
"slug": "lakehouse",
"sub_category_id": 1026,
"typical_lifespan": "EVERGREEN",
"volatility": "EMERGING"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "React Frontend Development",
"id": 96,
"rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
"slug": "d_init_01",
"source": "db"
},
"input_skill": "Lakehouse",
"llm_role": null,
"roles_from_db": []
}
],
"input_skill": "Lakehouse",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "Delta Lake",
"alias_type": "CANONICAL",
"id": 498,
"is_primary": true,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 13,
"display_name": "Delta Lake",
"id": 237,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "TOOL",
"slug": "delta-lake",
"sub_category_id": 1170,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Model and Data Versioning",
"id": 48,
"rationale": "Versioning systems for datasets, features, and model artifacts at the storage layer. This enables reproducible training, rollback, lineage of artifacts, and controlled promotion of model assets.",
"slug": "model-and-data-versioning",
"source": "db"
},
"input_skill": "Delta Lake",
"llm_role": null,
"roles_from_db": [
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
}
]
}
],
"input_skill": "Delta Lake",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "Apache Spark",
"alias_type": "CANONICAL",
"id": 2004,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "apache spark 3",
"alias_type": "VERSION",
"id": 2006,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "spark",
"alias_type": "VERSION",
"id": 2510,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "spark 3",
"alias_type": "VERSION",
"id": 2007,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "spark 3.x",
"alias_type": "VERSION",
"id": 2009,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "spark3",
"alias_type": "VERSION",
"id": 2008,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 5,
"display_name": "Apache Spark",
"id": 1350,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "FRAMEWORK",
"slug": "apache-spark",
"sub_category_id": 1021,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "ETL and ELT Tooling",
"id": 24,
"rationale": "Packaged tools for extracting, loading, and transforming data across systems. This dimension covers connector-based ingestion, transformation frameworks, and managed integration products.",
"slug": "etl-and-elt-tooling",
"source": "db"
},
"input_skill": "PySpark",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
}
],
"input_skill": "PySpark",
"matched_via": "embedding_alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [],
"canonical": null,
"dimensions": [],
"input_skill": "SparkSQL",
"matched_via": null,
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": {
"derived": {
"category": "Data Engineering Tools",
"skill_nature": "LANGUAGE",
"sub_category": "general",
"typical_lifespan": "MULTI_YEAR",
"version_strategy": "UNVERSIONED",
"volatility": "MEDIUM"
},
"enrichment": null,
"keep_log": [],
"locked_dimensions": [],
"merge_log": [],
"placed": null,
"relationships": null,
"skill_id": "sparksql",
"split_log": [],
"typed": null,
"warnings": []
},
"source_tag": "llm",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "Airflow",
"alias_type": "CANONICAL",
"id": 526,
"is_primary": true,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "airflow 2",
"alias_type": "VERSION",
"id": 2477,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "airflow-2",
"alias_type": "VERSION",
"id": 2478,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "airflow2",
"alias_type": "VERSION",
"id": 2476,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "airflow2.x",
"alias_type": "VERSION",
"id": 2479,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
},
{
"alias_text": "apache airflow 2",
"alias_type": "VERSION",
"id": 2480,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 13,
"display_name": "Airflow",
"id": 265,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "TOOL",
"slug": "airflow",
"sub_category_id": 130,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Workflow Orchestration for ML Pipelines",
"id": 54,
"rationale": "Workflow engines used to coordinate training, evaluation, deployment, and retraining jobs. This cluster covers dependencies, retries, scheduling, and pipeline composition for ML lifecycle automation.",
"slug": "workflow-orchestration-for-ml-pipelines",
"source": "db"
},
"input_skill": "Airflow",
"llm_role": null,
"roles_from_db": [
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
}
]
}
],
"input_skill": "Airflow",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [],
"canonical": null,
"dimensions": [],
"input_skill": "Big Data",
"matched_via": null,
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": {
"derived": {
"category": "Concepts",
"skill_nature": "CONCEPT",
"sub_category": "general",
"typical_lifespan": "EVERGREEN",
"version_strategy": "UNVERSIONED",
"volatility": "STABLE"
},
"enrichment": null,
"keep_log": [],
"locked_dimensions": [],
"merge_log": [],
"placed": null,
"relationships": null,
"skill_id": "big-data",
"split_log": [],
"typed": null,
"warnings": []
},
"source_tag": "llm",
"was_in_llm_skills": true
},
{
"aliases_in_db": [],
"canonical": null,
"dimensions": [],
"input_skill": "Data Warehousing",
"matched_via": null,
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": {
"derived": {
"category": "Concepts",
"skill_nature": "CONCEPT",
"sub_category": "general",
"typical_lifespan": "EVERGREEN",
"version_strategy": "UNVERSIONED",
"volatility": "STABLE"
},
"enrichment": null,
"keep_log": [],
"locked_dimensions": [],
"merge_log": [],
"placed": null,
"relationships": null,
"skill_id": "data-warehousing",
"split_log": [],
"typed": null,
"warnings": []
},
"source_tag": "llm",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "CI/CD",
"alias_type": "CANONICAL",
"id": 1826,
"is_primary": false,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 8,
"display_name": "CI/CD",
"id": 1190,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "METHODOLOGY",
"slug": "ci-cd",
"sub_category_id": 900,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "CI/CD Pipeline Platforms",
"id": 150,
"rationale": "Systems used to define, run, and maintain automated build and deployment workflows. This cluster is coherent because the role owns delivery automation end to end, including pipeline reliability and promotion logic.",
"slug": "ci-cd-pipeline-platforms",
"source": "db"
},
"input_skill": "CI/CD",
"llm_role": null,
"roles_from_db": [
{
"display_name": "DevOps Engineer",
"id": 10,
"rationale": null,
"role_archetype": null,
"slug": "devops-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "CI/CD for Machine Learning",
"id": 56,
"rationale": "Tools and platforms for automating ML model integration, testing, and deployment pipelines.",
"slug": "ci-cd-for-machine-learning",
"source": "db"
},
"input_skill": "CI/CD",
"llm_role": null,
"roles_from_db": [
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
}
]
}
],
"input_skill": "CI/CD",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "Snowflake",
"alias_type": "CANONICAL",
"id": 299,
"is_primary": true,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 9,
"display_name": "Snowflake",
"id": 105,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "PLATFORM",
"slug": "snowflake",
"sub_category_id": 113,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Data Warehouses",
"id": 22,
"rationale": "Managed analytical storage and compute platforms used for curated datasets, reporting, and downstream analytics. These systems are central to data modeling, performance tuning, and cost-aware query design.",
"slug": "cloud-data-warehouses",
"source": "db"
},
"input_skill": "Snowflake",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
}
],
"input_skill": "Snowflake",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "Terraform",
"alias_type": "CANONICAL",
"id": 547,
"is_primary": true,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 13,
"display_name": "Terraform",
"id": 286,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "TOOL",
"slug": "terraform",
"sub_category_id": 191,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Infrastructure \u0026 Security Automation Frameworks",
"id": 249,
"rationale": "Frameworks and libraries for provisioning, configuring, and automating cloud security infrastructure.",
"slug": "infrastructure-security-automation-frameworks",
"source": "db"
},
"input_skill": "Terraform",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Cloud Security Engineer",
"id": 23,
"rationale": null,
"role_archetype": null,
"slug": "cloud-security-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Infrastructure as Code",
"id": 132,
"rationale": "Declarative provisioning and environment definition tools used to codify cloud infrastructure, repeatable environments, and platform standards. Cloud Architects use these to express reference architectures and guardrails.",
"slug": "infrastructure-as-code",
"source": "db"
},
"input_skill": "Terraform",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Cloud Architect",
"id": 9,
"rationale": null,
"role_archetype": null,
"slug": "cloud-architect",
"source": "db"
},
{
"display_name": "DevOps Engineer",
"id": 10,
"rationale": null,
"role_archetype": null,
"slug": "devops-engineer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Infrastructure as Code for ML",
"id": 57,
"rationale": "Tools for provisioning and managing ML infrastructure resources through code.",
"slug": "infrastructure-as-code-for-ml",
"source": "db"
},
"input_skill": "Terraform",
"llm_role": null,
"roles_from_db": [
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
}
]
}
],
"input_skill": "Terraform",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [],
"canonical": null,
"dimensions": [],
"input_skill": "Large Language Models",
"matched_via": null,
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": {
"derived": {
"category": "Machine Learning Frameworks",
"skill_nature": "CONCEPT",
"sub_category": "general",
"typical_lifespan": "SHORT_LIVED",
"version_strategy": "VERSIONED",
"volatility": "FAST"
},
"enrichment": null,
"keep_log": [],
"locked_dimensions": [],
"merge_log": [],
"placed": null,
"relationships": null,
"skill_id": "large-language-models",
"split_log": [],
"typed": null,
"warnings": []
},
"source_tag": "llm",
"was_in_llm_skills": true
},
{
"aliases_in_db": [],
"canonical": null,
"dimensions": [],
"input_skill": "AWS Glue",
"matched_via": null,
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": {
"derived": {
"category": "Cloud Platforms",
"skill_nature": "PLATFORM",
"sub_category": "general",
"typical_lifespan": "MULTI_YEAR",
"version_strategy": "UNVERSIONED",
"volatility": "MEDIUM"
},
"enrichment": null,
"keep_log": [],
"locked_dimensions": [],
"merge_log": [],
"placed": null,
"relationships": null,
"skill_id": "aws-glue",
"split_log": [],
"typed": null,
"warnings": []
},
"source_tag": "llm",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "Amazon S3",
"alias_type": "CANONICAL",
"id": 379,
"is_primary": true,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 11,
"display_name": "Amazon S3",
"id": 170,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "CLOUD_SERVICE",
"slug": "amazon-s3",
"sub_category_id": 120,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Storage and Data Services",
"id": 144,
"rationale": "Cloud-native storage and managed data services used to place workloads, choose durability tiers, and define platform boundaries. This is a coherent cluster because architects evaluate storage fit, access patterns, and managed service tradeoffs.",
"slug": "cloud-storage-and-data-services",
"source": "db"
},
"input_skill": "Amazon S3",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Cloud Architect",
"id": 9,
"rationale": null,
"role_archetype": null,
"slug": "cloud-architect",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Storage and File Formats",
"id": 35,
"rationale": "Object storage and data file formats used as the physical substrate for data movement and lake-style analytics. Data engineers need these to manage landing zones, partitioned datasets, and efficient interchange.",
"slug": "cloud-storage-and-file-formats",
"source": "db"
},
"input_skill": "Amazon S3",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
}
],
"input_skill": "Amazon S3",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "Amazon SQS",
"alias_type": "CANONICAL",
"id": 176,
"is_primary": true,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 11,
"display_name": "Amazon SQS",
"id": 39,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "CLOUD_SERVICE",
"slug": "amazon-sqs",
"sub_category_id": 56,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Messaging and Background Jobs",
"id": 291,
"rationale": "Asynchronous processing patterns and worker systems used to decouple backend work from request handling. This is a coherent cluster because the role supports background jobs, retries, and deferred processing.",
"slug": "messaging-and-background-jobs",
"source": "db"
},
"input_skill": "Amazon SQS",
"llm_role": null,
"roles_from_db": [
{
"display_name": "PHP Backend Developer",
"id": 86,
"rationale": null,
"role_archetype": "Engineering",
"slug": "php-backend-developer",
"source": "db"
},
{
"display_name": "Python Backend Developer",
"id": 80,
"rationale": null,
"role_archetype": "Engineering",
"slug": "python-backend-developer",
"source": "db"
},
{
"display_name": "Ruby Backend Developer",
"id": 85,
"rationale": null,
"role_archetype": "Engineering",
"slug": "ruby-backend-developer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Messaging and Event Streaming",
"id": 8,
"rationale": "Transport-layer systems used to move events and decouple producers from consumers. Data engineers use these systems to ingest, buffer, and distribute event data before downstream processing.",
"slug": "messaging-and-event-streaming",
"source": "db"
},
"input_skill": "Amazon SQS",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Backend Developer",
"id": 1,
"rationale": null,
"role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
"slug": "backend-engineer",
"source": "db"
},
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
]
}
],
"input_skill": "Amazon SQS",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "AWS SNS",
"alias_type": "CANONICAL",
"id": 5151,
"is_primary": true,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 11,
"display_name": "AWS SNS",
"id": 3579,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "CLOUD_SERVICE",
"slug": "aws-sns",
"sub_category_id": 2847,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Asynchronous Messaging and Event Streaming",
"id": 297,
"rationale": "Asynchronous communication patterns and broker technologies used to decouple backend services and move work off the request path. Includes queues, pub/sub, event streams, consumer groups, dead-letter queues, and delivery semantics across systems such as Kafka, RabbitMQ, NATS, SQS/SNS, Pulsar, and ActiveMQ.",
"slug": "asynchronous-messaging-and-event-streaming",
"source": "db"
},
"input_skill": "Amazon SNS",
"llm_role": null,
"roles_from_db": [
{
"display_name": ".NET Backend Developer",
"id": 83,
"rationale": null,
"role_archetype": "Engineering",
"slug": "dotnet-backend-developer",
"source": "db"
},
{
"display_name": "Go Backend Developer",
"id": 81,
"rationale": null,
"role_archetype": "Engineering",
"slug": "go-backend-developer",
"source": "db"
},
{
"display_name": "Kotlin Backend Developer",
"id": 84,
"rationale": null,
"role_archetype": "Engineering",
"slug": "kotlin-server-backend-developer",
"source": "db"
},
{
"display_name": "Node.js Backend Developer",
"id": 82,
"rationale": null,
"role_archetype": "Engineering",
"slug": "node-backend-developer",
"source": "db"
},
{
"display_name": "Scala Backend Developer",
"id": 87,
"rationale": null,
"role_archetype": "Engineering",
"slug": "scala-backend-developer",
"source": "db"
}
]
}
],
"input_skill": "Amazon SNS",
"matched_via": "embedding_alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
},
{
"aliases_in_db": [
{
"alias_text": "AWS Lambda",
"alias_type": "CANONICAL",
"id": 3563,
"is_primary": true,
"match_strategy": "CASE_INSENSITIVE"
}
],
"canonical": {
"category_id": 11,
"display_name": "AWS Lambda",
"id": 2295,
"is_also_category": false,
"is_extractable": true,
"skill_nature": "CLOUD_SERVICE",
"slug": "aws-lambda",
"sub_category_id": 1704,
"typical_lifespan": "EVERGREEN",
"volatility": "STABLE"
},
"dimensions": [
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Platforms",
"id": 20,
"rationale": "Underlying cloud providers that host the managed services or infrastructure used by the role, such as AWS, Azure, and GCP.",
"slug": "cloud-platforms",
"source": "db"
},
"input_skill": "AWS Lambda",
"llm_role": null,
"roles_from_db": [
{
"display_name": ".NET Backend Developer",
"id": 83,
"rationale": null,
"role_archetype": "Engineering",
"slug": "dotnet-backend-developer",
"source": "db"
},
{
"display_name": "Backend Developer",
"id": 1,
"rationale": null,
"role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
"slug": "backend-engineer",
"source": "db"
},
{
"display_name": "Cyber Security Engineer",
"id": 5,
"rationale": null,
"role_archetype": null,
"slug": "cybersecurity-engineer",
"source": "db"
},
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
},
{
"display_name": "DevOps Engineer",
"id": 10,
"rationale": null,
"role_archetype": null,
"slug": "devops-engineer",
"source": "db"
},
{
"display_name": "Fullstack Developer",
"id": 15,
"rationale": null,
"role_archetype": null,
"slug": "full-stack-engineer",
"source": "db"
},
{
"display_name": "Go Backend Developer",
"id": 81,
"rationale": null,
"role_archetype": "Engineering",
"slug": "go-backend-developer",
"source": "db"
},
{
"display_name": "Java Backend Developer",
"id": 79,
"rationale": null,
"role_archetype": "Engineering",
"slug": "java-backend-developer",
"source": "db"
},
{
"display_name": "Kotlin Backend Developer",
"id": 84,
"rationale": null,
"role_archetype": "Engineering",
"slug": "kotlin-server-backend-developer",
"source": "db"
},
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
},
{
"display_name": "Node.js Backend Developer",
"id": 82,
"rationale": null,
"role_archetype": "Engineering",
"slug": "node-backend-developer",
"source": "db"
},
{
"display_name": "Python Backend Developer",
"id": 80,
"rationale": null,
"role_archetype": "Engineering",
"slug": "python-backend-developer",
"source": "db"
},
{
"display_name": "Scala Backend Developer",
"id": 87,
"rationale": null,
"role_archetype": "Engineering",
"slug": "scala-backend-developer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Platforms \u0026 Hosting Providers",
"id": 278,
"rationale": "Familiarity with vendor-specific hosting and backend services for deploying and scaling web applications.",
"slug": "cloud-platforms-hosting-providers",
"source": "db"
},
"input_skill": "AWS Lambda",
"llm_role": null,
"roles_from_db": [
{
"display_name": ".NET Backend Developer",
"id": 83,
"rationale": null,
"role_archetype": "Engineering",
"slug": "dotnet-backend-developer",
"source": "db"
},
{
"display_name": "Kotlin Backend Developer",
"id": 84,
"rationale": null,
"role_archetype": "Engineering",
"slug": "kotlin-server-backend-developer",
"source": "db"
},
{
"display_name": "Scala Backend Developer",
"id": 87,
"rationale": null,
"role_archetype": "Engineering",
"slug": "scala-backend-developer",
"source": "db"
},
{
"display_name": "Web Developer",
"id": 25,
"rationale": null,
"role_archetype": null,
"slug": "web-developer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Platforms \u0026 Managed Services",
"id": 221,
"rationale": "Operates and integrates vendor-specific cloud compute, storage, and hosting services.",
"slug": "cloud-platforms-managed-services",
"source": "db"
},
"input_skill": "AWS Lambda",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Fullstack Developer",
"id": 15,
"rationale": null,
"role_archetype": null,
"slug": "full-stack-engineer",
"source": "db"
},
{
"display_name": "Go Backend Developer",
"id": 81,
"rationale": null,
"role_archetype": "Engineering",
"slug": "go-backend-developer",
"source": "db"
},
{
"display_name": "Node.js Backend Developer",
"id": 82,
"rationale": null,
"role_archetype": "Engineering",
"slug": "node-backend-developer",
"source": "db"
}
]
},
{
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Platforms \u0026 Services",
"id": 296,
"rationale": "Deploying, configuring, and managing Python backends on major cloud and PaaS offerings.",
"slug": "cloud-platforms-services",
"source": "db"
},
"input_skill": "AWS Lambda",
"llm_role": null,
"roles_from_db": [
{
"display_name": "Python Backend Developer",
"id": 80,
"rationale": null,
"role_archetype": "Engineering",
"slug": "python-backend-developer",
"source": "db"
}
]
}
],
"input_skill": "AWS Lambda",
"matched_via": "alias",
"new_alias_persisted": false,
"new_alias_text": null,
"new_skill_meta": null,
"source_tag": "db",
"was_in_llm_skills": true
}
],
"unmatched_skills": [
"Apache NiFi",
"SparkSQL",
"Big Data",
"Data Warehousing",
"Large Language Models",
"AWS Glue"
]
}
API 3 — final-role-output
{
"chosen_role": {
"display_name": "Data Engineer",
"id": 2,
"rationale": "Domain=Data Engineering \u0026 Analytics; The JD is centered on Spark, Databricks, AWS, Airflow, big data, and lakehouse/data warehousing work, which most closely matches Data Engineer.",
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
},
"chosen_role_resolution": "in_db",
"final_input_skills": [
{
"skill": "Spark",
"tag": "in_db"
},
{
"skill": "SQL",
"tag": "in_db"
},
{
"skill": "Databricks",
"tag": "in_db"
},
{
"skill": "AWS",
"tag": "in_db"
},
{
"skill": "Apache NiFi",
"tag": "new"
},
{
"skill": "Lakehouse",
"tag": "in_db"
},
{
"skill": "Delta Lake",
"tag": "in_db"
},
{
"skill": "PySpark",
"tag": "in_db"
},
{
"skill": "SparkSQL",
"tag": "new"
},
{
"skill": "Airflow",
"tag": "in_db"
},
{
"skill": "Big Data",
"tag": "new"
},
{
"skill": "Data Warehousing",
"tag": "new"
},
{
"skill": "CI/CD",
"tag": "in_db"
},
{
"skill": "Snowflake",
"tag": "in_db"
},
{
"skill": "Terraform",
"tag": "in_db"
},
{
"skill": "Large Language Models",
"tag": "new"
},
{
"skill": "AWS Glue",
"tag": "new"
},
{
"skill": "Amazon S3",
"tag": "in_db"
},
{
"skill": "Amazon SQS",
"tag": "in_db"
},
{
"skill": "Amazon SNS",
"tag": "in_db"
},
{
"skill": "AWS Lambda",
"tag": "in_db"
}
],
"llm_cost_api1_usd": null,
"llm_cost_api2_usd": null,
"llm_cost_api3_usd": null,
"llm_cost_total_usd": null,
"persistence": {
"items": [
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "ETL and ELT Tooling",
"id": 24,
"rationale": "Packaged tools for extracting, loading, and transforming data across systems. This dimension covers connector-based ingestion, transformation frameworks, and managed integration products.",
"slug": "etl-and-elt-tooling",
"source": "db"
},
"dimension_id": 24,
"input_skill": "Spark",
"llm_role": null,
"matched_chosen_role": true,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
"role_dimension_saved": true,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 1350,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Pega Programming Languages \u0026 DSLs",
"id": 267,
"rationale": "Programming languages and domain-specific languages used in Pega development.",
"slug": "pega-programming-languages-dsls",
"source": "db"
},
"dimension_id": 267,
"input_skill": "SQL",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Pega Developer",
"id": 24,
"rationale": null,
"role_archetype": null,
"slug": "pega-developer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 101,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages \u0026 DSLs",
"id": 475,
"rationale": "Oversee and guide the selection and effective use of programming and domain\u2010specific languages in software projects.",
"slug": "programming-languages-dsls",
"source": "db"
},
"dimension_id": 475,
"input_skill": "SQL",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Engineering Manager",
"id": 121,
"rationale": null,
"role_archetype": null,
"slug": "engineering-manager",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 101,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Programming Languages for Data Work",
"id": 21,
"rationale": "Languages used to implement data pipelines, transformations, and operational glue. This is the primary coding surface for building ingestion, enrichment, and automation logic in data engineering.",
"slug": "programming-languages-for-data-work",
"source": "db"
},
"dimension_id": 21,
"input_skill": "SQL",
"llm_role": null,
"matched_chosen_role": true,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
"role_dimension_saved": true,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 101,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "React Frontend Development",
"id": 96,
"rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
"slug": "d_init_01",
"source": "db"
},
"dimension_id": 96,
"input_skill": "Databricks",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [],
"skill_dimension_saved": true,
"skill_id": 1202,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Platforms",
"id": 20,
"rationale": "Underlying cloud providers that host the managed services or infrastructure used by the role, such as AWS, Azure, and GCP.",
"slug": "cloud-platforms",
"source": "db"
},
"dimension_id": 20,
"input_skill": "AWS",
"llm_role": null,
"matched_chosen_role": true,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
"role_dimension_saved": true,
"roles_from_db": [
{
"display_name": ".NET Backend Developer",
"id": 83,
"rationale": null,
"role_archetype": "Engineering",
"slug": "dotnet-backend-developer",
"source": "db"
},
{
"display_name": "Backend Developer",
"id": 1,
"rationale": null,
"role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
"slug": "backend-engineer",
"source": "db"
},
{
"display_name": "Cyber Security Engineer",
"id": 5,
"rationale": null,
"role_archetype": null,
"slug": "cybersecurity-engineer",
"source": "db"
},
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
},
{
"display_name": "DevOps Engineer",
"id": 10,
"rationale": null,
"role_archetype": null,
"slug": "devops-engineer",
"source": "db"
},
{
"display_name": "Fullstack Developer",
"id": 15,
"rationale": null,
"role_archetype": null,
"slug": "full-stack-engineer",
"source": "db"
},
{
"display_name": "Go Backend Developer",
"id": 81,
"rationale": null,
"role_archetype": "Engineering",
"slug": "go-backend-developer",
"source": "db"
},
{
"display_name": "Java Backend Developer",
"id": 79,
"rationale": null,
"role_archetype": "Engineering",
"slug": "java-backend-developer",
"source": "db"
},
{
"display_name": "Kotlin Backend Developer",
"id": 84,
"rationale": null,
"role_archetype": "Engineering",
"slug": "kotlin-server-backend-developer",
"source": "db"
},
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
},
{
"display_name": "Node.js Backend Developer",
"id": 82,
"rationale": null,
"role_archetype": "Engineering",
"slug": "node-backend-developer",
"source": "db"
},
{
"display_name": "Python Backend Developer",
"id": 80,
"rationale": null,
"role_archetype": "Engineering",
"slug": "python-backend-developer",
"source": "db"
},
{
"display_name": "Scala Backend Developer",
"id": 87,
"rationale": null,
"role_archetype": "Engineering",
"slug": "scala-backend-developer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 187,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Platforms for AI Deployment",
"id": 211,
"rationale": "Major cloud services that provide infrastructure and managed services for AI workloads.",
"slug": "cloud-platforms-for-ai-deployment",
"source": "db"
},
"dimension_id": 211,
"input_skill": "AWS",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "AI Engineer",
"id": 13,
"rationale": null,
"role_archetype": null,
"slug": "ai-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 187,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Provider Platforms",
"id": 131,
"rationale": "Major cloud platforms and their core service ecosystems used to design target-state architectures, choose deployment boundaries, and evaluate managed capabilities. This is the primary substrate for cloud architecture decisions.",
"slug": "cloud-provider-platforms",
"source": "db"
},
"dimension_id": 131,
"input_skill": "AWS",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Cloud Architect",
"id": 9,
"rationale": null,
"role_archetype": null,
"slug": "cloud-architect",
"source": "db"
},
{
"display_name": "Cloud Security Engineer",
"id": 23,
"rationale": null,
"role_archetype": null,
"slug": "cloud-security-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 187,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Security Posture Tools",
"id": 64,
"rationale": "Cloud-native security platforms used to assess misconfiguration, workload exposure, and cloud control coverage. This dimension includes the major CNAPP/CSPM/CWPP vendors and cloud security services the role reviews and tunes.",
"slug": "cloud-security-posture-tools",
"source": "db"
},
"dimension_id": 64,
"input_skill": "AWS",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Cloud Security Engineer",
"id": 23,
"rationale": null,
"role_archetype": null,
"slug": "cloud-security-engineer",
"source": "db"
},
{
"display_name": "Cyber Security Engineer",
"id": 5,
"rationale": null,
"role_archetype": null,
"slug": "cybersecurity-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 187,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Vendor Product Families",
"id": 477,
"rationale": "Coordinate usage, licensing, and architecture decisions for major vendor software and cloud product families.",
"slug": "vendor-product-families",
"source": "db"
},
"dimension_id": 477,
"input_skill": "AWS",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Engineering Manager",
"id": 121,
"rationale": null,
"role_archetype": null,
"slug": "engineering-manager",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 187,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "React Frontend Development",
"id": 96,
"rationale": "Building interactive web user interfaces with React.js, including component composition, state management, hooks, and rendering patterns. React.js belongs here because it is a core library for client-side UI development in modern web applications.",
"slug": "d_init_01",
"source": "db"
},
"dimension_id": 96,
"input_skill": "Lakehouse",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [],
"skill_dimension_saved": true,
"skill_id": 1359,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Model and Data Versioning",
"id": 48,
"rationale": "Versioning systems for datasets, features, and model artifacts at the storage layer. This enables reproducible training, rollback, lineage of artifacts, and controlled promotion of model assets.",
"slug": "model-and-data-versioning",
"source": "db"
},
"dimension_id": 48,
"input_skill": "Delta Lake",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 237,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "ETL and ELT Tooling",
"id": 24,
"rationale": "Packaged tools for extracting, loading, and transforming data across systems. This dimension covers connector-based ingestion, transformation frameworks, and managed integration products.",
"slug": "etl-and-elt-tooling",
"source": "db"
},
"dimension_id": 24,
"input_skill": "PySpark",
"llm_role": null,
"matched_chosen_role": true,
"outcome_line": "Skipped \u2014 no persistable v3 meta for new skill",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
],
"skill_dimension_saved": false,
"skill_id": null,
"skill_tag": "new",
"skipped_reason": "skill_not_in_db_v3_proposed"
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Workflow Orchestration for ML Pipelines",
"id": 54,
"rationale": "Workflow engines used to coordinate training, evaluation, deployment, and retraining jobs. This cluster covers dependencies, retries, scheduling, and pipeline composition for ML lifecycle automation.",
"slug": "workflow-orchestration-for-ml-pipelines",
"source": "db"
},
"dimension_id": 54,
"input_skill": "Airflow",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 265,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "CI/CD Pipeline Platforms",
"id": 150,
"rationale": "Systems used to define, run, and maintain automated build and deployment workflows. This cluster is coherent because the role owns delivery automation end to end, including pipeline reliability and promotion logic.",
"slug": "ci-cd-pipeline-platforms",
"source": "db"
},
"dimension_id": 150,
"input_skill": "CI/CD",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "DevOps Engineer",
"id": 10,
"rationale": null,
"role_archetype": null,
"slug": "devops-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 1190,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "CI/CD for Machine Learning",
"id": 56,
"rationale": "Tools and platforms for automating ML model integration, testing, and deployment pipelines.",
"slug": "ci-cd-for-machine-learning",
"source": "db"
},
"dimension_id": 56,
"input_skill": "CI/CD",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 1190,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Data Warehouses",
"id": 22,
"rationale": "Managed analytical storage and compute platforms used for curated datasets, reporting, and downstream analytics. These systems are central to data modeling, performance tuning, and cost-aware query design.",
"slug": "cloud-data-warehouses",
"source": "db"
},
"dimension_id": 22,
"input_skill": "Snowflake",
"llm_role": null,
"matched_chosen_role": true,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
"role_dimension_saved": true,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 105,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Infrastructure \u0026 Security Automation Frameworks",
"id": 249,
"rationale": "Frameworks and libraries for provisioning, configuring, and automating cloud security infrastructure.",
"slug": "infrastructure-security-automation-frameworks",
"source": "db"
},
"dimension_id": 249,
"input_skill": "Terraform",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Cloud Security Engineer",
"id": 23,
"rationale": null,
"role_archetype": null,
"slug": "cloud-security-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 286,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Infrastructure as Code",
"id": 132,
"rationale": "Declarative provisioning and environment definition tools used to codify cloud infrastructure, repeatable environments, and platform standards. Cloud Architects use these to express reference architectures and guardrails.",
"slug": "infrastructure-as-code",
"source": "db"
},
"dimension_id": 132,
"input_skill": "Terraform",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Cloud Architect",
"id": 9,
"rationale": null,
"role_archetype": null,
"slug": "cloud-architect",
"source": "db"
},
{
"display_name": "DevOps Engineer",
"id": 10,
"rationale": null,
"role_archetype": null,
"slug": "devops-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 286,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Infrastructure as Code for ML",
"id": 57,
"rationale": "Tools for provisioning and managing ML infrastructure resources through code.",
"slug": "infrastructure-as-code-for-ml",
"source": "db"
},
"dimension_id": 57,
"input_skill": "Terraform",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 286,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Storage and Data Services",
"id": 144,
"rationale": "Cloud-native storage and managed data services used to place workloads, choose durability tiers, and define platform boundaries. This is a coherent cluster because architects evaluate storage fit, access patterns, and managed service tradeoffs.",
"slug": "cloud-storage-and-data-services",
"source": "db"
},
"dimension_id": 144,
"input_skill": "Amazon S3",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Cloud Architect",
"id": 9,
"rationale": null,
"role_archetype": null,
"slug": "cloud-architect",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 170,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Storage and File Formats",
"id": 35,
"rationale": "Object storage and data file formats used as the physical substrate for data movement and lake-style analytics. Data engineers need these to manage landing zones, partitioned datasets, and efficient interchange.",
"slug": "cloud-storage-and-file-formats",
"source": "db"
},
"dimension_id": 35,
"input_skill": "Amazon S3",
"llm_role": null,
"matched_chosen_role": true,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
"role_dimension_saved": true,
"roles_from_db": [
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 170,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Messaging and Background Jobs",
"id": 291,
"rationale": "Asynchronous processing patterns and worker systems used to decouple backend work from request handling. This is a coherent cluster because the role supports background jobs, retries, and deferred processing.",
"slug": "messaging-and-background-jobs",
"source": "db"
},
"dimension_id": 291,
"input_skill": "Amazon SQS",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "PHP Backend Developer",
"id": 86,
"rationale": null,
"role_archetype": "Engineering",
"slug": "php-backend-developer",
"source": "db"
},
{
"display_name": "Python Backend Developer",
"id": 80,
"rationale": null,
"role_archetype": "Engineering",
"slug": "python-backend-developer",
"source": "db"
},
{
"display_name": "Ruby Backend Developer",
"id": 85,
"rationale": null,
"role_archetype": "Engineering",
"slug": "ruby-backend-developer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 39,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Messaging and Event Streaming",
"id": 8,
"rationale": "Transport-layer systems used to move events and decouple producers from consumers. Data engineers use these systems to ingest, buffer, and distribute event data before downstream processing.",
"slug": "messaging-and-event-streaming",
"source": "db"
},
"dimension_id": 8,
"input_skill": "Amazon SQS",
"llm_role": null,
"matched_chosen_role": true,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
"role_dimension_saved": true,
"roles_from_db": [
{
"display_name": "Backend Developer",
"id": 1,
"rationale": null,
"role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
"slug": "backend-engineer",
"source": "db"
},
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 39,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Asynchronous Messaging and Event Streaming",
"id": 297,
"rationale": "Asynchronous communication patterns and broker technologies used to decouple backend services and move work off the request path. Includes queues, pub/sub, event streams, consumer groups, dead-letter queues, and delivery semantics across systems such as Kafka, RabbitMQ, NATS, SQS/SNS, Pulsar, and ActiveMQ.",
"slug": "asynchronous-messaging-and-event-streaming",
"source": "db"
},
"dimension_id": 297,
"input_skill": "Amazon SNS",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Skipped \u2014 no persistable v3 meta for new skill",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": ".NET Backend Developer",
"id": 83,
"rationale": null,
"role_archetype": "Engineering",
"slug": "dotnet-backend-developer",
"source": "db"
},
{
"display_name": "Go Backend Developer",
"id": 81,
"rationale": null,
"role_archetype": "Engineering",
"slug": "go-backend-developer",
"source": "db"
},
{
"display_name": "Kotlin Backend Developer",
"id": 84,
"rationale": null,
"role_archetype": "Engineering",
"slug": "kotlin-server-backend-developer",
"source": "db"
},
{
"display_name": "Node.js Backend Developer",
"id": 82,
"rationale": null,
"role_archetype": "Engineering",
"slug": "node-backend-developer",
"source": "db"
},
{
"display_name": "Scala Backend Developer",
"id": 87,
"rationale": null,
"role_archetype": "Engineering",
"slug": "scala-backend-developer",
"source": "db"
}
],
"skill_dimension_saved": false,
"skill_id": null,
"skill_tag": "new",
"skipped_reason": "skill_not_in_db_v3_proposed"
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Platforms",
"id": 20,
"rationale": "Underlying cloud providers that host the managed services or infrastructure used by the role, such as AWS, Azure, and GCP.",
"slug": "cloud-platforms",
"source": "db"
},
"dimension_id": 20,
"input_skill": "AWS Lambda",
"llm_role": null,
"matched_chosen_role": true,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension saved",
"role_dimension_saved": true,
"roles_from_db": [
{
"display_name": ".NET Backend Developer",
"id": 83,
"rationale": null,
"role_archetype": "Engineering",
"slug": "dotnet-backend-developer",
"source": "db"
},
{
"display_name": "Backend Developer",
"id": 1,
"rationale": null,
"role_archetype": "A Backend Engineer designs, builds, and maintains the server-side logic and data handling that power applications and services. They focus on implementing reliable business functionality, integrating with other systems, and ensuring the backend is scalable, maintainable, and observable.",
"slug": "backend-engineer",
"source": "db"
},
{
"display_name": "Cyber Security Engineer",
"id": 5,
"rationale": null,
"role_archetype": null,
"slug": "cybersecurity-engineer",
"source": "db"
},
{
"display_name": "Data Engineer",
"id": 2,
"rationale": null,
"role_archetype": null,
"slug": "data-engineer",
"source": "db"
},
{
"display_name": "DevOps Engineer",
"id": 10,
"rationale": null,
"role_archetype": null,
"slug": "devops-engineer",
"source": "db"
},
{
"display_name": "Fullstack Developer",
"id": 15,
"rationale": null,
"role_archetype": null,
"slug": "full-stack-engineer",
"source": "db"
},
{
"display_name": "Go Backend Developer",
"id": 81,
"rationale": null,
"role_archetype": "Engineering",
"slug": "go-backend-developer",
"source": "db"
},
{
"display_name": "Java Backend Developer",
"id": 79,
"rationale": null,
"role_archetype": "Engineering",
"slug": "java-backend-developer",
"source": "db"
},
{
"display_name": "Kotlin Backend Developer",
"id": 84,
"rationale": null,
"role_archetype": "Engineering",
"slug": "kotlin-server-backend-developer",
"source": "db"
},
{
"display_name": "ML Engineer",
"id": 3,
"rationale": null,
"role_archetype": null,
"slug": "ml-engineer",
"source": "db"
},
{
"display_name": "MLOps Engineer",
"id": 16,
"rationale": null,
"role_archetype": null,
"slug": "ml-ops-engineer",
"source": "db"
},
{
"display_name": "Node.js Backend Developer",
"id": 82,
"rationale": null,
"role_archetype": "Engineering",
"slug": "node-backend-developer",
"source": "db"
},
{
"display_name": "Python Backend Developer",
"id": 80,
"rationale": null,
"role_archetype": "Engineering",
"slug": "python-backend-developer",
"source": "db"
},
{
"display_name": "Scala Backend Developer",
"id": 87,
"rationale": null,
"role_archetype": "Engineering",
"slug": "scala-backend-developer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 2295,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Platforms \u0026 Hosting Providers",
"id": 278,
"rationale": "Familiarity with vendor-specific hosting and backend services for deploying and scaling web applications.",
"slug": "cloud-platforms-hosting-providers",
"source": "db"
},
"dimension_id": 278,
"input_skill": "AWS Lambda",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": ".NET Backend Developer",
"id": 83,
"rationale": null,
"role_archetype": "Engineering",
"slug": "dotnet-backend-developer",
"source": "db"
},
{
"display_name": "Kotlin Backend Developer",
"id": 84,
"rationale": null,
"role_archetype": "Engineering",
"slug": "kotlin-server-backend-developer",
"source": "db"
},
{
"display_name": "Scala Backend Developer",
"id": 87,
"rationale": null,
"role_archetype": "Engineering",
"slug": "scala-backend-developer",
"source": "db"
},
{
"display_name": "Web Developer",
"id": 25,
"rationale": null,
"role_archetype": null,
"slug": "web-developer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 2295,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Platforms \u0026 Managed Services",
"id": 221,
"rationale": "Operates and integrates vendor-specific cloud compute, storage, and hosting services.",
"slug": "cloud-platforms-managed-services",
"source": "db"
},
"dimension_id": 221,
"input_skill": "AWS Lambda",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Fullstack Developer",
"id": 15,
"rationale": null,
"role_archetype": null,
"slug": "full-stack-engineer",
"source": "db"
},
{
"display_name": "Go Backend Developer",
"id": 81,
"rationale": null,
"role_archetype": "Engineering",
"slug": "go-backend-developer",
"source": "db"
},
{
"display_name": "Node.js Backend Developer",
"id": 82,
"rationale": null,
"role_archetype": "Engineering",
"slug": "node-backend-developer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 2295,
"skill_tag": "in_db",
"skipped_reason": null
},
{
"chosen_role_id": 2,
"dimension": {
"difficulty_hint": "well_known",
"display_name": "Cloud Platforms \u0026 Services",
"id": 296,
"rationale": "Deploying, configuring, and managing Python backends on major cloud and PaaS offerings.",
"slug": "cloud-platforms-services",
"source": "db"
},
"dimension_id": 296,
"input_skill": "AWS Lambda",
"llm_role": null,
"matched_chosen_role": false,
"outcome_line": "Existing dimension (library) \u00b7 Role\u2194dimension skipped (dimension not under chosen role)",
"role_dimension_saved": false,
"roles_from_db": [
{
"display_name": "Python Backend Developer",
"id": 80,
"rationale": null,
"role_archetype": "Engineering",
"slug": "python-backend-developer",
"source": "db"
}
],
"skill_dimension_saved": true,
"skill_id": 2295,
"skill_tag": "in_db",
"skipped_reason": null
}
],
"new_skills_created": 0,
"role_dimension_saved": 0,
"skill_dimension_saved": 0,
"skipped": 2
},
"planner_output": null,
"run_id": "216829da-da8a-4849-996b-1fd470c98354"
}