Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .claude/settings.local.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"permissions": {
"allow": [
"mcp__context7__resolve-library-id",
"mcp__context7__get-library-docs",
"Bash(mkdir:*)"
],
"deny": [],
"ask": []
}
}
Comment on lines +1 to +11
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this does not belong here. It's local.

238 changes: 238 additions & 0 deletions karate-example/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,238 @@
# Karate Testomat.io Integration Example

This example demonstrates how to properly integrate Karate tests with Testomat.io using unique test IDs for Scenario Outline examples.

## Overview

This project contains a simple Karate test that demonstrates:
- Testing the Swagger Petstore API (https://petstore.swagger.io/v2/)
- Proper use of unique `@T...` test IDs in Scenario Outline examples
- Correct integration with Testomat.io for individual test tracking

## Features

### Test Scenarios
1. **Get pet by ID** - Tests retrieving pets with different IDs (existing and non-existing)
2. **Add new pet** - Tests creating new pets with different data

### Unique Test IDs
Each Scenario Outline example has a unique test ID in the format `@T[8 alphanumeric characters]`:
- `T1a2b3c4d`, `T5e6f7g8h`, `T9i0j1k2l` (GET scenarios)
- `Tm3n4o5p6`, `Tq7r8s9t0`, `Tu1v2w3x4` (POST scenarios)

## 🚀 Setup Framework From Scratch (In a Nutshell)

### Quick Setup (5 minutes)

```bash
# 1. Create project directory
mkdir my-karate-project
cd my-karate-project

# 2. Create Maven directory structure
mkdir -p src/test/java/com/example/karate
mkdir -p src/test/resources

# 3. Create pom.xml with Karate dependencies
cat > pom.xml << 'EOF'
<?xml version="1.0" encoding="UTF-8"?>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need to do things like that?

<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<artifactId>karate-project</artifactId>
<version>1.0.0</version>
<properties>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<karate.version>1.4.1</karate.version>
</properties>
<dependencies>
<dependency>
<groupId>com.intuit.karate</groupId>
<artifactId>karate-junit5</artifactId>
<version>${karate.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<testResources>
<testResource>
<directory>src/test/java</directory>
<excludes><exclude>**/*.java</exclude></excludes>
</testResource>
</testResources>
</build>
</project>
EOF

# 4. Create test runner
cat > src/test/java/com/example/karate/TestRunner.java << 'EOF'
package com.example.karate;
import com.intuit.karate.Results;
import com.intuit.karate.Runner;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertTrue;

public class TestRunner {
@Test
void testAll() {
Results results = Runner.path("classpath:")
.outputJunitXml(true)
.parallel(1);
assertTrue(results.getFailCount() == 0, results.getErrorMessages());
}
}
EOF

# 5. Create your first .feature file
cat > src/test/resources/sample-test.feature << 'EOF'
Feature: Sample API Test

Background:
* url 'https://petstore.swagger.io/v2'

Scenario Outline: Get pet - <testCase> @<testId>
Given path 'pet', <petId>
When method get
Then status <expectedStatus>

Examples:
| testCase | petId | expectedStatus | testId |
| existing | 1 | 200 | T1a2b3c4d |
| nonexistent | 999 | 404 | T5e6f7g8h |
EOF

# 6. Run your first test
mvn test
```
Comment on lines +23 to +110
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As it's a demo project that will not be or will be rarely updated, I would rather do this using Maven Archetype or, simpler, tell the user to clone the project with a ready-to-use structure. Both ways are more user-friendly and faster to use.


### That's it! You now have:
✅ Working Karate framework
✅ Sample API test with unique test IDs
✅ JUnit XML reports in `target/karate-reports/`
✅ Ready for Testomat.io integration

### Next Steps
```bash
# Install report-xml tool (if not installed)
npm install -g @testomatio/reporter

# Import to Testomat.io
TESTOMATIO=your_key TESTOMATIO_TITLE_IDS=1 npx report-xml "target/karate-reports/**/*.xml"
```

## Prerequisites

- Java 11+
- Maven 3.6+
- Internet connection (to access Petstore API)
- Node.js (for Testomat.io import)

## Running Tests

### Local Execution

```bash
# Run all tests
mvn test

# Run specific test class
mvn test -Dtest=KarateTest

# Run with specific Karate options
mvn test -Dkarate.options="--tags @T1a2b3c4d"
```

### Alternative Feature File Execution

```bash
# Run specific feature file directly
mvn test -Dkarate.options="classpath:simple-api-test.feature"
```

## Testomat.io Integration

### 1. Generate JUnit XML Reports

Tests automatically generate JUnit XML reports in `target/karate-reports/` when executed.

### 2. Import to Testomat.io

```bash
# Import test results using unique test IDs
TESTOMATIO=your_project_key TESTOMATIO_TITLE_IDS=1 npx report-xml "target/karate-reports/**/*.xml"
```

### Key Integration Points

- **TESTOMATIO_TITLE_IDS=1**: Ensures Testomat.io uses the `@T...` IDs from scenario titles
- **Unique Test IDs**: Each example in Scenario Outline has a unique `@T...` ID
- **Proper Naming**: Scenario titles include `@<testId>` placeholder

## Project Structure

```
karate-example/
├── pom.xml
├── README.md
└── src/
└── test/
├── java/
│ └── com/
│ └── testomat/
│ └── karate/
│ └── KarateTest.java
└── resources/
└── simple-api-test.feature
```

Comment on lines +176 to +191
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this structure refers to the project the user creates in the first part of the README, then it's wrong. If it does not then as a user I can't see where I get this from.

## API Details

### Swagger Petstore API
- **Base URL**: https://petstore.swagger.io/v2
- **Endpoints Used**:
- `GET /pet/{petId}` - Retrieve pet by ID
- `POST /pet` - Add new pet
- **Expected Behavior**:
- Pet IDs 1-10 typically exist (status 200)
- Pet ID 999+ typically don't exist (status 404)
- POST requests create new pets (status 200)

## Test ID Format

Test IDs follow the pattern: `T` + 8 alphanumeric characters
- Must be unique within the project
- Used in scenario titles via `@<testId>` placeholder
- Enables proper test tracking in Testomat.io

## Troubleshooting

### Common Issues

1. **Tests fail due to network**: Ensure internet access to petstore.swagger.io
2. **XML reports not generated**: Check `target/karate-reports/` directory exists
3. **Testomat.io import issues**: Verify `TESTOMATIO_TITLE_IDS=1` is set

### Verification

1. **Run tests successfully**: `mvn test` should show 6 scenarios passed
2. **Check XML reports**: Verify unique test case names contain `@T...` IDs
3. **Testomat.io import**: Each scenario should appear as separate test

## Example Output

When tests run successfully, you should see:
```
6 scenarios (6 passed)
```

JUnit XML will contain entries like:
```xml
<testcase name="Get pet by ID - Pet 1 @T1a2b3c4d" .../>
<testcase name="Get pet by ID - Pet 2 @T5e6f7g8h" .../>
```

This ensures each test appears separately in Testomat.io rather than as retries of the same test.
74 changes: 74 additions & 0 deletions karate-example/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">

<modelVersion>4.0.0</modelVersion>
<groupId>com.testomat</groupId>
<artifactId>karate-example</artifactId>
<version>1.0.0</version>
<packaging>jar</packaging>

<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<java.version>11</java.version>
<maven.compiler.version>3.8.1</maven.compiler.version>
<maven.surefire.version>3.0.0-M7</maven.surefire.version>
<karate.version>1.4.1</karate.version>
<junit.version>5.8.2</junit.version>
</properties>

<dependencies>
<dependency>
<groupId>com.intuit.karate</groupId>
<artifactId>karate-junit5</artifactId>
<version>${karate.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<testResources>
<testResource>
<directory>src/test/java</directory>
<excludes>
<exclude>**/*.java</exclude>
</excludes>
</testResource>
<testResource>
<directory>src/test/resources</directory>
</testResource>
</testResources>

<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven.compiler.version}</version>
<configuration>
<encoding>UTF-8</encoding>
<source>${java.version}</source>
<target>${java.version}</target>
<compilerArgument>-Werror</compilerArgument>
</configuration>
</plugin>

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven.surefire.version}</version>
<configuration>
<argLine>-Dfile.encoding=UTF-8</argLine>
</configuration>
</plugin>
</plugins>
</build>

</project>
17 changes: 17 additions & 0 deletions karate-example/src/test/java/com/testomat/karate/KarateTest.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
package com.testomat.karate;

import com.intuit.karate.Results;
import com.intuit.karate.Runner;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertTrue;

public class KarateTest {

@Test
void testAll() {
Results results = Runner.path("classpath:simple-api-test.feature")
.outputJunitXml(true)
.parallel(1);
assertTrue(results.getFailCount() == 0, results.getErrorMessages());
}
}
40 changes: 40 additions & 0 deletions karate-example/src/test/resources/simple-api-test.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
Feature: Pet Store API Testing

# This feature demonstrates proper Karate-Testomat.io integration
# Each Scenario Outline example has a unique @T... ID for proper test separation

Background:
* url 'https://petstore.swagger.io/v2'

Scenario Outline: Get pet by ID - Pet <petId> @<testId>
# Test retrieving pets with different IDs and expected status codes
Given path 'pet', <petId>
When method get
Then status <expectedStatus>
# Log successful retrievals for debugging
* if (<expectedStatus> == 200) karate.log('Pet found:', response.name)
# Log not found cases for debugging
* if (<expectedStatus> == 404) karate.log('Pet not found for ID:', <petId>)

Examples:
| petId | expectedStatus | testId |
| 1 | 200 | T1a2b3c4d |
| 2 | 200 | T5e6f7g8h |
| 999 | 404 | T9i0j1k2l |

Scenario Outline: Add new pet - <petName> @<testId>
# Test creating new pets with different data
* def petData = { "id": <petId>, "name": "<petName>", "status": "available" }
Given path 'pet'
And request petData
When method post
Then status <expectedStatus>
# Validate response contains the pet data
* if (<expectedStatus> == 200) match response.name == '<petName>'
* if (<expectedStatus> == 200) match response.id == <petId>

Examples:
| petId | petName | expectedStatus | testId |
| 1001 | Buddy | 200 | Tm3n4o5p6 |
| 1002 | Rex | 200 | Tq7r8s9t0 |
| 1003 | Fluffy | 200 | Tu1v2w3x4 |
Loading