Skip to content

Added code snippet markers to agent responses #43920

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 89 commits into
base: main
Choose a base branch
from
Open

Added code snippet markers to agent responses #43920

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
47a79e3
Copy code from https://github.com/Azure/azure-sdk-for-python-pr/tree/…
dargilco Oct 30, 2025
8a0fbe3
Update emitter-package.json
dargilco Oct 30, 2025
72d6a64
exclude all other packages from ci.yml
dargilco Oct 30, 2025
9a58fbc
Update cspell.json
dargilco Oct 30, 2025
ec726c6
Update broken link in README.md
dargilco Oct 30, 2025
3b48a65
Fix one Agent sample. Fix broken link in CHANGELOG.md
dargilco Oct 30, 2025
1630e5b
Address Copilot review comments
dargilco Oct 30, 2025
ed53bf7
Fix spelling in test code
dargilco Oct 30, 2025
b6ce9ad
Update pip install command in samples to `azure-ai-projects>=2.0.0b1`
dargilco Oct 30, 2025
3889417
Update post-emitter-fixes.cmd
dargilco Oct 30, 2025
75e0506
Re-emit from latst TypeSpec
dargilco Oct 30, 2025
bc523bf
Update post-emitter-fixes.cmd
dargilco Oct 30, 2025
ab70711
Remove old get_openai_client method
dargilco Oct 30, 2025
d8cdba4
Move Responses samples to a new folder. Update package README.md
dargilco Oct 31, 2025
d06b5b6
update ci.yml
dargilco Oct 31, 2025
6b4dd29
Merge remote-tracking branch 'origin/main' into feature/azure-ai-proj…
dargilco Oct 31, 2025
0a1a7d4
Shorter method names for Memory Store and Agent Manifest operations (…
dargilco Oct 31, 2025
234a749
Re-emit from TypeSpec (#43731)
dargilco Oct 31, 2025
2c9f1d0
fix
w-javed Nov 3, 2025
7ea6598
fix
w-javed Nov 3, 2025
a890b07
revert
w-javed Nov 3, 2025
d1255bf
Merge remote-tracking branch 'origin/main' into feature/azure-ai-proj…
dargilco Nov 3, 2025
634eeeb
Re-emit from latest TypeSpec. Includes some TypeSpec doc string updat…
dargilco Nov 3, 2025
ac68bf3
Merge branch 'feature/azure-ai-projects/2.0.0b1' into evaluation_samp…
w-javed Nov 3, 2025
329041e
fix
w-javed Nov 3, 2025
5eaee2f
adding responses instrumentation (#43762)
M-Hietala Nov 3, 2025
ff8cc56
Add Samples for Agentic Evaluators in azure-ai-projects (#43769)
m7md7sien Nov 3, 2025
f501b3e
Disable doc generation (sphinx) for now, until the tool is fixed
dargilco Nov 3, 2025
3c2acbe
Merge branch 'feature/azure-ai-projects/2.0.0b1' of https://github.co…
dargilco Nov 3, 2025
5eda9e5
adding more tracing (#43768)
M-Hietala Nov 3, 2025
76cf69c
Re-emit from TypeSpec
dargilco Nov 4, 2025
594d5d5
Merge branch 'feature/azure-ai-projects/2.0.0b1' of https://github.co…
dargilco Nov 4, 2025
6433272
adding custom eval
w-javed Nov 4, 2025
7f499ff
Merge branch 'evaluation_samples_graders' of https://github.com/Azure…
w-javed Nov 4, 2025
8408068
fix
w-javed Nov 4, 2025
b594612
fix
w-javed Nov 4, 2025
b0ce73d
Fix all evaluators
w-javed Nov 4, 2025
8241eee
fix
w-javed Nov 4, 2025
88d409d
Merge remote-tracking branch 'origin/main' into feature/azure-ai-proj…
dargilco Nov 4, 2025
0d7977c
add sample for trace eval (#43782)
ninghu Nov 4, 2025
19ef60c
Added Red Teaming Evaluation, Evaluation Taxonomy, Scheduled RedTeami…
posaninagendra Nov 4, 2025
f812d87
workflow samples (#43785)
howieleung Nov 4, 2025
aa4b941
Improvements to console logging (#43780)
dargilco Nov 4, 2025
1b145aa
Re-emit from TypeSpec, to rename WorkflowDefinition to WorkflowAgentD…
dargilco Nov 4, 2025
362fba0
Merge branch 'feature/azure-ai-projects/2.0.0b1' of https://github.co…
dargilco Nov 4, 2025
8393660
Updated workflow samples to use new name WorkflowAgentDefinition
dargilco Nov 4, 2025
dbbac66
Merge remote-tracking branch 'origin/main' into feature/azure-ai-proj…
dargilco Nov 4, 2025
3871804
Aprilk/agent target insights samples (#43792)
aprilk-ms Nov 4, 2025
02e59db
updated (#43795)
posaninagendra Nov 4, 2025
309b4a7
Sample updates (#43767)
howieleung Nov 5, 2025
4653d95
Fix pylint errors
dargilco Nov 5, 2025
f08b77c
fix
w-javed Nov 5, 2025
261c992
fix name
w-javed Nov 5, 2025
fceab57
Migrate samples from agentsv2-preview repo (#43798)
howieleung Nov 5, 2025
ac079d2
[Agent Evaluator Samples]: Add Conversation Example in Relevance and …
m7md7sien Nov 5, 2025
14e77ad
Fix
w-javed Nov 5, 2025
8be0fa3
Merge branch 'evaluation_samples_graders' of https://github.com/Azure…
w-javed Nov 5, 2025
19ba455
Format and rename samples (#43819)
howieleung Nov 5, 2025
cc9b09a
adding tool call instrumentation (#43799)
M-Hietala Nov 5, 2025
be3f29f
Add samples for "responses" without using AIProjectClient (#43820)
dargilco Nov 6, 2025
38038e0
define recording decorators that can record openai traffic within tes…
scbedd Nov 6, 2025
17c996f
additional conversation items support (#43836)
M-Hietala Nov 6, 2025
fb1727c
Fix tool call accuracy sample (#43847)
salma-elshafey Nov 6, 2025
7010582
Modify sample agent response evaluation (#43849)
salma-elshafey Nov 6, 2025
4561ed4
updated prompt based custom evaluation
vebudumu Nov 6, 2025
4712be1
Fix emitted enum value A2_A_PREVIEW to A2A_PREVIEW (#43853)
dargilco Nov 6, 2025
94fd5a5
adding ai assisted
w-javed Nov 7, 2025
b59c992
adding output
w-javed Nov 7, 2025
99a2d0d
adding output
w-javed Nov 7, 2025
dfa92c7
Re-emit from latest TypeSpec, resolve some arch board review comments…
dargilco Nov 7, 2025
15da461
updates to binary data tracing (#43854)
M-Hietala Nov 7, 2025
08161f6
Found certain cases isinstance not reliable for annotaiton. So I now…
howieleung Nov 7, 2025
60aa09f
Fix structured output in SDK and add samples and tests (#43868)
dargilco Nov 7, 2025
398835f
fix
w-javed Nov 7, 2025
1464d5d
Merge remote-tracking branch 'origin/main' into feature/azure-ai-proj…
dargilco Nov 7, 2025
de57163
adding comments secion for prompt based
w-javed Nov 7, 2025
8aad922
adding image generation tracing support (#43883)
M-Hietala Nov 7, 2025
aa7162a
Hookup test recording for OpenAI Responses tests (#43882)
dargilco Nov 7, 2025
d10d223
samples update (#43879)
howieleung Nov 8, 2025
7618dbf
Add samples/responses/sample_responses_structured_output.py
dargilco Nov 8, 2025
dad7bf7
Merge branch 'feature/azure-ai-projects/2.0.0b1' of https://github.co…
dargilco Nov 8, 2025
25bb909
Try to fix OpenAI responses tests
dargilco Nov 8, 2025
46236ba
Update recording assets
dargilco Nov 8, 2025
14c1d13
Disable Responses tests again. Also take care of slash at the end of …
dargilco Nov 8, 2025
9838d30
spell
w-javed Nov 8, 2025
fe93cb4
resolved
w-javed Nov 8, 2025
6340d2a
black fixes
w-javed Nov 8, 2025
0528016
Added code snippet markers
aprilk-ms Nov 10, 2025
dfac735
Updated readme on using Evals from OpenAI client
aprilk-ms Nov 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion eng/emitter-package-lock.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion eng/emitter-package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
"@azure-tools/typespec-autorest": "~0.61.0",
"@azure-tools/typespec-azure-rulesets": "~0.61.0",
"@azure-tools/typespec-client-generator-core": "~0.61.0",
"@azure-tools/typespec-liftr-base": "0.10.0"
"@azure-tools/typespec-liftr-base": "0.10.0",
"azure-ai-decorators": "https://github.com/glecaros/pkg/raw/refs/heads/main/azure-ai-decorators-v0.1.2.tgz"
}
}
3 changes: 2 additions & 1 deletion eng/tools/azure-sdk-tools/devtools_testutils/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@
set_session_recording_options,
)
from .cert import create_combined_bundle
from .helpers import ResponseCallback, RetryCounter, is_live_and_not_recording, trim_kwargs_from_test_function
from .helpers import ResponseCallback, RetryCounter, is_live_and_not_recording, is_live as is_live_internal, trim_kwargs_from_test_function
from .fake_credentials import FakeTokenCredential

PowerShellPreparer = EnvironmentVariableLoader # Backward compat
Expand Down Expand Up @@ -117,4 +117,5 @@
"FakeTokenCredential",
"create_combined_bundle",
"is_live_and_not_recording",
"is_live_internal"
]
28 changes: 27 additions & 1 deletion sdk/ai/azure-ai-projects/.env.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,38 @@
# 3. Run the tests (`pytest`) or run samples in the `samples` folder
#

#######################################################################
#
# Used in samples
#
# Project endpoint has the format:
# `https://<your-ai-services-account-name>.services.ai.azure.com/api/projects/<your-project-name>`
AZURE_AI_PROJECT_ENDPOINT=
AZURE_AI_MODEL_DEPLOYMENT_NAME=
AGENT_NAME=
CONVERSATION_ID=
CONNECTION_NAME=

# Used in tests
#######################################################################
#
# Used tests, excluding Agent tests
AZURE_AI_PROJECTS_TESTS_PROJECT_ENDPOINT=

# Used in Agent tests
AZURE_AI_PROJECTS_TESTS_AGENTS_PROJECT_ENDPOINT=

# Used in tracing tests
AZURE_AI_PROJECTS_TESTS_TRACING_PROJECT_ENDPOINT=

# Used in Container App Agents tests
AZURE_AI_PROJECTS_TESTS_CONTAINER_PROJECT_ENDPOINT=
AZURE_AI_PROJECTS_TESTS_CONTAINER_APP_RESOURCE_ID=
AZURE_AI_PROJECTS_TESTS_CONTAINER_INGRESS_SUBDOMAIN_SUFFIX=

# Used in tools
BING_PROJECT_CONNECTION_ID=
MCP_PROJECT_CONNECTION_ID=




Loading