> ## Documentation Index
> Fetch the complete documentation index at: https://docs.odigos.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Enrich with OpenTelemetry API

export const language_0 = "Python";

Odigos will automatically instrument your {language_0} services and record semantic spans from popular modules. Many users find the automatic instrumentation data sufficient for their needs.

However, if there is anything specific to your application that you want to record, you can enrich the data by adding custom spans to your code.

This is sometimes referred to as `manual instrumentation`.

## Use Cases

Examples of custom spans you might want to add to your code include:

* Spans for the execution of some potentially heavy or slow computation in you service.
* Tracing for internal or third party libraries for which you don't have automatic or integrated instrumentation.
* Spans to describe some logical operations in your business logic that are meaningful in your domain.

## Required Dependencies

Install the API from PyPI using pip:

```bash theme={null}
pip install opentelemetry-api
```

## Creating Spans

To create a span, use the `tracer` object from the `opentelemetry.trace` module. The `tracer` object is a factory for creating spans.

```python theme={null}
from opentelemetry import trace

tracer = trace.get_tracer(__name__)

def my_function():
   with tracer.start_as_current_span("my_function") as span:
      print("Hello world!")
```

Important Notes:

1. **Assign meaningful names to spans**:
   Use descriptive names for spans, (such as the function name) to clearly describe the operations they represent. This helps anyone examining the trace to easily understand the span's purpose and context.
2. **Avoid dynamic, high cardinality data in span names**:
   Do not include dynamic data such as IDs in the span name, as this can cause performance issues and make the trace harder to read. Instead, use static, descriptive names for spans and record dynamic information in span attributes. This ensures better performance and readability of the trace.

### Recording Span Attributes

Span attributes are key-value pairs that record additional information about an operation, which can be useful for debugging, performance analysis, or troubleshooting

Examples:

* User ID, organization ID, Account ID or other identifiers.
* Inputs - the relevant parameters or configuration that influenced the operation.
* Outputs - the result of the operation.
* Entities - the entities or resources that the operation interacted with.

Attribute names are lowercased strings in the form `my_application.my_attribute`, example: `my_service.user_id`.
Read more [here](https://opentelemetry.io/docs/specs/semconv/general/attribute-naming/)

To record attributes, use the `set_attribute` method on the span object.

```python theme={null}
def my_function(arg: str):
   with tracer.start_as_current_span("my_function") as span:
      span.set_attribute("argument_name", arg)
```

Important Notes:

1. **Be cautious when recording data**:
   Avoid including PII (personally identifiable information) or any data you do not wish to expose in your traces.
2. **Attribute cost considerations**:
   Each attribute affects performance and processing time. Record only what is necessary and avoid superfluous data.
3. **Use static names for attributes**:
   Avoid dynamic content such as IDs in attribute keys. Use static names and properly namespace them (scope.attribute\_name) to provide clear context for downstream consumers.
4. **Adhere to OpenTelemetry semantic conventions**:
   Prefer using namespaces and attribute names from the OpenTelemetry semantic conventions to enhance data interoperability and make it easier for others to understand.

### Recording Errors

To easily identify and monitor errors in your traces, the Span object includes a status field that can be used to mark the span as an error. This helps in spotting errors in trace viewers, sampling, and setting up alerts.

If an operation you are tracing fails, you can mark the span's status as an error and record the error details within the span. Here's how you can do it:

* An exception has been raised to demonstrate an error that occurred in your code.

```python theme={null}
def my_function():
   with tracer.start_as_current_span("my_function") as span:
      try:
         print("Hello world!")
         raise Exception("Some Exception")
      except Exception as e:
         span.record_exception(e)
         span.set_status(trace.Status(trace.StatusCode.ERROR, str(e)))
```

## Advanced Capabilities

The Odigos Python agent extends the standard OpenTelemetry SDK with capabilities that apply to **all spans** — including custom spans you create with the API above. These features are configured remotely through OpAMP and take effect without restarting your application.

### Head Sampling

The Python agent supports [head sampling](../../pipeline/sampling/head-and-tail-sampling#head-sampling-agent) at the root span. Odigos pushes [Noisy Operation](../../pipeline/sampling/rules/noisy-operation) rules to the agent, which evaluates them through a custom `OdigosSampler` wired into the `TracerProvider`.

Custom spans participate in the same sampling pipeline as auto-instrumented spans:

* **Root spans** — If your custom span is the entry point of a trace (no active parent), head-sampling rules can match it and decide whether the entire trace is kept or dropped.
* **Child spans** — Follow parent-based sampling: if the parent trace is sampled, custom child spans are recorded; if not, they are dropped (or recorded for metrics only — see [Span metrics mode](#span-metrics-mode) below).
* **Context propagation** — Use `tracer.start_as_current_span(...)` (as shown above) so custom spans attach to the active trace and inherit the sampling decision.

Head-sampling rules match HTTP server spans (`SpanKind.SERVER`) and HTTP client spans (`SpanKind.CLIENT`) using OpenTelemetry semantic conventions. The sampler accepts both stable and legacy attribute names — for example `http.request.method` and `http.method`, `server.address` and `http.host`. Server rules match on route (with `{param}` wildcard support), route prefix, and method. Client rules match on server address, templated path, path prefix, and method.

When multiple rules match, the **lowest keep percentage** wins. Even when a trace is dropped, the agent attaches an `odigos` [tracestate](https://www.w3.org/TR/trace-context/#tracestate-header) entry so downstream services and the Odigos tail sampler can coordinate on the same decision.

For rule configuration and evaluation order, see [Sampling rules overview](../../pipeline/sampling/rules/overview).

### Dry Run Mode

When cluster-wide dry run (`samplingDryRun` in [OdigosConfiguration](../../api-reference/odigos.io.v1alpha1)) is enabled, the Python agent still exports every span but annotates the sampling decision in tracestate (`dry:t` if the trace would have been kept, `dry:f` if it would have been dropped). Use dry run to validate Noisy rules before committing to real drops.

### Span Metrics Mode

The global `spanMetricsMode` setting controls what happens to spans that head sampling marks as not sampled:

| Mode                           | Not-sampled spans                                                                                   |
| ------------------------------ | --------------------------------------------------------------------------------------------------- |
| `sampled-spans-only` (default) | Dropped entirely — no trace export, no span metrics                                                 |
| `all-spans`                    | Recorded with `RECORD_ONLY` — excluded from trace export but forwarded for span metrics computation |

This applies to custom spans the same way it applies to auto-instrumented spans. If you rely on span metrics for RED dashboards, set `spanMetricsMode: all-spans` in your Odigos configuration so unsampled traffic still contributes to rate and error metrics.

### Agent Diagnostics

Debug logging for the Odigos Python agent and its OpenTelemetry components can be toggled at runtime through an [InstrumentationRule](../../api-reference/odigos.io.v1alpha1#odigos-io-v1alpha1-InstrumentationRule) with `agentDiagnostics`:

```yaml theme={null}
spec:
  agentDiagnostics:
    odigosLogLevel: debug
    openTelemetryComponentsLogLevel: debug
```

Supported levels are `error`, `warn`, `info`, and `debug`. Remote config from OpAMP **fully overrides** the startup environment variables — if a level is not set in the remote config, that logger is silenced.

For the window before the first OpAMP message arrives, you can set startup log levels with environment variables:

| Variable           | Logger                       | Values                                                     |
| ------------------ | ---------------------------- | ---------------------------------------------------------- |
| `ODIGOS_LOG_LEVEL` | Odigos agent (`odigos`)      | `ALL`, `VERBOSE`, `DEBUG`, `INFO`, `WARN`, `ERROR`, `NONE` |
| `OTEL_LOG_LEVEL`   | OpenTelemetry SDK components | Same as above                                              |

At `debug` level, the sampler logs each rule evaluation — useful when troubleshooting why a custom or auto-instrumented span was kept or dropped.

## Additional Information

For more use cases, see the [OpenTelemetry Python API documentation](https://opentelemetry.io/docs/languages/python/instrumentation/#creating-spans).