Product Releases

​

What’s New and Improved

• Set custom bins at model creation or update via model.schema["column"].bins = [...] and model.update(), or through the UI schema editor
• Bins must be strictly increasing, span [min, max], with at most 16 boundary values (15 bins)
• When omitted, 10 uniform bins are auto-generated (default behavior unchanged)
• Updating bins triggers re-computation of historical aggregates; models with more than 10M events per environment are not eligible for bin updates
• See Customizing Your Model Schema

• min(x) and max(x) aggregate functions: Compute the minimum or maximum value of a numeric column or expression across all rows in a time window. Use them to track feature or attribute ranges (max(col) - min(col)), detect extreme values, or build ratio-based metrics (min(col) / max(col))
  • For ML: accepts column references (e.g., min(Prediction))
  • For agentic/GenAI: accepts attribute() calls (e.g., max(attribute('response_time_ms', type='user', scope='span')))

• null keyword: The null literal can now be used directly as a return value in FQL expressions — for example, in if() to propagate missing data: if(is_null(col), null, col * 2). Use is_null() and is_not_null() to filter by null values

• Improved Error States: We now give more helpful information if the user viewing the dashboard has no access to projects or if the projects they can access have no onboarded models, instead of displaying a generic error message as we were previously doing.

• Named access keys: Assign a descriptive name to each key at creation so you can quickly identify which integration or environment uses it.
• Configurable limit: Each user can create up to 5 keys by default. Contact Fiddler Support to request a higher limit.

​

What’s New and Improved

• PDF export: Renders the full dashboard at a consistent page width, scales content to fit, and produces a multi-page PDF when the content is taller than a single page
• PNG export: Captures the visible dashboard as a single high-resolution image
• Smart exclusions: UI chrome such as control panels, date pickers, and fullscreen toggles are excluded from the export so the output contains only chart content
• Force-load: Before capturing, the exporter pre-fetches all chart data and waits for charts to finish rendering, so exports are never blank or partially loaded
• Scope: Available on all dashboard surfaces when the enable_dashboard_export_ui feature flag is enabled

• Dynamic bulk action label: The button reads “Hide All” when all series are visible, “Show All” when all are hidden, and “Invert Selection” when the chart has a mix of visible and hidden series
• Double-click to isolate: Double-clicking a legend item hides every other series and shows only the clicked one, making it easy to focus on a single metric
  • Benefit: Quickly toggle visibility of many series at once or drill into a single series without clicking each legend item individually
  • Scope: Applies to both Monitoring and GenAI Application charts when more than three legend items are present

• Works with Filters Panel: RCA chart will be filtered by the intersection of filter selections between the legend and the filters panel in the sidebar

• FQL-based definitions: Write expressions like average(attribute('gen_ai.usage.input_tokens', type='system', scope='span')) to define metrics over any span attribute — system or user-defined
• Flexible scoping: Create organization-wide global metrics or scope them to specific projects for team-level visibility
• Full lifecycle management: Create, view, and delete custom metrics directly from the Fiddler UI.
• Use cases: Track cost per span, compute custom latency percentiles, measure token usage ratios, or aggregate any numeric span attribute over time
  • Benefit: Define the exact measurements that matter to your agentic applications without waiting for built-in metric support
  • Scope: Available for all GenAI Applications

• Improved Jailbreaking Detection: Enhanced accuracy for identifying jailbreaking attempts in prompts
• Improved Roleplaying Detection: Enhanced accuracy for detecting content where the model is instructed to adopt a specific persona or character
  • Note: No changes to the model interface — works as a drop-in replacement for the previous version

• Model ID: fiddler/ministral3-8b
• Use case: Complex or nuanced evaluations that benefit from stronger reasoning capabilities
• Trade-off: Slower than fiddler/llama3.1-8b but better suited for evaluations requiring deeper analysis
• Benefit: Run sophisticated evaluators without configuring external provider credentials or incurring per-token charges

• Syntax: gini(actual=Target, predicted=Score) — both actual and predicted are required keyword arguments of type Number
• All ML task types: Works for binary classification, multiclass classification, regression, and ranking models
• Normalized Gini: Compute the normalized Gini coefficient (scaled to 1 for a perfect model) with gini(actual=Target, predicted=Score) / gini(actual=Target, predicted=Target)
• AUC shortcut: For binary classification, the normalized Gini can also be derived as 2 * auroc() - 1
• Categorical targets: For binary classification models with categorical target columns, use an if expression to convert to numeric (e.g. gini(actual=if(churn == 'yes', 1, 0), predicted=score))
• Benefit: Compute Gini-based model performance metrics — widely used in insurance and credit risk modeling — directly in custom metric definitions, charts, and alerts
• Scope: Available for ML custom metrics only (not for GenAI or agentic applications). See Fiddler Query Language and Custom Metrics for full documentation

​

What’s New and Improved

• Benefit: You can update multiple target columns (e.g., comment and feedback) in a single PATCH request without data loss
• Scope: Affects only LLM and NOT_SET model task types, which are the only tasks that support multiple targets. Classification, regression, and ranking models (single target) are unaffected

• Benefit: You can manage evaluation datasets entirely from the UI without writing code; smart auto-mapping reduces friction for repeated uploads
• Note: CSV uploads are limited to 1,000 rows per upload

​

Fixes and Security Updates

• LLM Gateway: Provider delete and update operations now return 409 Conflict instead of a generic 400 when credentials are referenced by evaluators
• Security Updates: Applied routine security patches and version updates to application frameworks to stay current
• Chart zoom and legend state preserved across re-renders: Zoom position and legend selection no longer reset when hovering over chart elements or when the chart re-renders. Previously, user-interactive chart state was discarded on every option update, causing charts to “jitter” and lose filter settings

​

What’s New and Improved

• KPI Summary Cards: At-a-glance metrics including total models, critical and warning alert counts, and monitoring coverage percentage
  • Benefit: You can quickly assess overall AI system health without navigating into individual projects
• Alert Breakdown by Type and Severity: Alert summary cards grouped by type (drift, performance, data integrity) showing affected models, top triggered metrics, and time since last trigger
  • Benefit: Identify which alert categories need attention and drill down to specific alerts with one click
• Cross-Project Model Health Table: A unified table listing all models across projects with severity status indicators, traffic volume, last event timestamp, and model owner
  • Benefit: Compare model health across the entire organization and quickly spot models that need investigation
• Configurable Time Range: Selectable time windows (7D, 14D, 30D, 90D) to control the reporting period for all dashboard metrics
  • Benefit: Adjust the analysis window to match your reporting cadence or investigate recent vs. long-term trends
• Model Fleet Health Donut Chart: Visual distribution of healthy, warning, critical, and unmonitored models across the organization
  • Benefit: Instantly understand the proportion of your model fleet in each health state

• Zero-configuration ingestion: Point OTEL_EXPORTER_OTLP_ENDPOINT at your Fiddler instance and set your application ID — traces flow in automatically
• Purpose-built span mapper: Handles LiteLLM proxy’s span format (acompletion operation name, JSON message attributes, infrastructure spans) and maps them to Fiddler’s semantic convention
• Full conversation capture: Extracts system prompt, last user turn, and assistant response from LiteLLM’s JSON message attributes (gen_ai.input.messages, gen_ai.output.messages)
• Cost and proxy metadata: LiteLLM cost fields (gen_ai.cost.*) and proxy metadata (metadata.*) are preserved as user-visible span attributes for auditing and cost attribution

• Drag-and-Drop Rearrangement: Reorder dashboard charts by dragging and dropping tiles into new positions
  • Benefit: Quickly customize your dashboard layout to prioritize the metrics that matter most
• Inline Resize: Resize chart tiles directly on the dashboard using drag handles
  • Benefit: Adjust chart sizes to give more space to complex visualizations without leaving the page
• Swap Behavior: When dragging a tile onto another, the tiles swap positions rather than overlapping
  • Benefit: Predictable layout behavior that preserves all your charts during rearrangement

• Benefit: You can quickly find the highest or lowest values for evaluator scores, custom metrics, and other numeric columns without manual scanning.

• Benefit: You can click directly from an RCA row to open source documents, external links, or trace URLs without copying and pasting.

• Send Test Alert Action: A new “Send Test Alert” option is available in the alert rules 3-dot menu
  • Benefit: Validate notification delivery without waiting for a real alert to trigger
• Backend API: New POST /v3/alert-rules/{id}/test-notification endpoint supports test notification delivery with org-scoping and input validation
  • Benefit: Programmatic access to test notifications for automation and CI/CD workflows

​

Fixes and Security Updates

• Alert Email Input: Email addresses typed into the alert notification recipient field are now automatically captured when the field loses focus, preventing silent data loss when the user clicks away without pressing Enter. Email validation logic has been consolidated into a shared webapp-common helper for consistent behavior across all alert configuration surfaces
• Security Updates: Applied routine security patches and version updates to application frameworks to stay up-to-date with the latest improvements.

​

What’s New and Improved

• Grid-Based Application View: A new card-based grid layout is available as an alternative to the existing list view
  • Benefit: Faster visual scanning and easier identification of applications, especially in environments with many GenAI applications
• Project-Level Administration: Projects are now exposed as a manageable entity within the admin console
  • Benefit: Administrators can manage project-level RBAC permissions and perform project deletion directly from the UI
  • Note: This enables customers to manage access control and lifecycle for projects without requiring backend intervention

• Unified Chart Controls: Reworked chart controls provide a consistent experience across all GenAI monitoring views
  • Benefit: Streamlined chart configuration with fewer clicks and a more intuitive layout
• Reworked Alert Controls: Alert configuration within GenAI experiences has been updated to align with the new chart controls
  • Benefit: Consistent interaction patterns between charting and alerting workflows
• Automatic Chart Migration: Existing GenAI charts are automatically migrated to the new format during the upgrade

• Hierarchical TTL Configuration: TTL values can be set at organization, project, or model scope by platform administrators using the config manager APIs
  • How it works: More specific levels take precedence — model-level TTL overrides project-level, which overrides organization-level
  • Unit: Days (minimum 7 days)
• Lazy Pruning: Data pruning occurs on a background schedule, at intervals of at least 4 hours
  • Note: Dashboards and historical charts are unaffected; only raw event queries are limited to data within the TTL window
• Opt-In by Default: Existing deployments are unaffected unless custom TTL values are explicitly configured
  • Note: The deployment-wide default is controlled by the EVENTS_TABLE_TTL_DAYS environment variable, which defaults to no TTL

• Size-Based Batching: Sending queues now use byte-based sizing instead of count-based
  • Benefit: More predictable batching behavior for payloads of varying sizes, especially large spans
• Percentage-Based Memory Limiter: The collector’s memory limiter is now percentage-based, adapting to container memory limits automatically
  • Benefit: Better suited to containerized deployments compared to the previous fixed-value approach
• Configurable Kafka Producer: Kafka producer max request size is now configurable with optional compression
  • Benefit: Supports larger trace and event payloads without hitting message size limits

​

Client Versions

​

Fixes and Security Updates

• Security Updates: Applied routine security patches and version updates to application frameworks to stay up-to-date with the latest improvements.

​

What’s New and Improved

• Answer Relevance 2.0 (Enhanced): Improved ordinal scoring system replaces binary scoring
• Scoring: High (1.0), Medium (0.5), Low (0.0) with detailed reasoning
• Benefit: Granular assessment of how well responses address user queries
• Availability: Agentic Monitoring, Experiments, LLM Observability
• Context Relevance (New): Measures whether retrieved documents are relevant to the query
• Scoring: High (1.0), Medium (0.5), Low (0.0) with detailed reasoning
• Benefit: Isolate retrieval problems from generation problems in your RAG pipeline
• Availability: Agentic Monitoring and Experiments (not available in LLM Observability)
• RAG Faithfulness (Repackaged): LLM-as-a-Judge faithfulness evaluator with standardized inputs and outputs, now part of the RAG Health Metrics triad
• Scoring: Binary — Yes (1.0) / No (0.0) with detailed reasoning
• Inputs: user_query, rag_response, retrieved_documents
• Outputs: label, value, reasoning
• Benefit: Combined with Answer Relevance and Context Relevance for comprehensive RAG pipeline diagnostics
• Availability: Agentic Monitoring, Experiments, LLM Observability
• Note: FTL Faithfulness (ftl_response_faithfulness) continues unchanged for guardrails and LLM Observability use cases
• Note: RAG Faithfulness and FTL Faithfulness are separate evaluators with different architectures

• Prompt Template Style: Define custom evaluators using prompt_template with Jinja {{ placeholder }} syntax and output_fields for structured evaluation results
• Benefit: Build domain-specific evaluation criteria for any use case — brand voice compliance, bias detection, topic classification, and more
• Note: This is the same CustomJudge class available in Agentic Monitoring, now also accessible through the Evals SDK for Experiments workflows

• Better Reasoning Output: More intelligent, consistent reasoning that better aligns with predicted labels
• Scope: All pre-built LLM-as-a-Judge evaluators in Agentic Monitoring, and custom LLM-as-a-Judge evaluators using Prompt Template-style prompt specs in Traditional Monitoring
• Benefit: More detailed and accurate reasoning explanations for evaluation results
• Note: Evaluation speed may be slightly slower in some cases due to higher quality reasoning output; tokens per second remains constant

• OpenTelemetry Upgrade: Updated to OpenTelemetry 1.39.1 / 0.60b1 (api, sdk, instrumentation, otlp exporter)
• Note: Removed hardcoded OpenTelemetry span/batch limits; SDK now uses OpenTelemetry defaults
• Full LLM Message History: New span attributes gen_ai.input.messages and gen_ai.output.messages capture complete LLM message history
• Benefit: Better debugging and observability of LLM interactions within agentic workflows

• Benefit: Enables observability and analysis of Strands-based agentic pipelines within the Fiddler platform

​

Client Versions

​

Deprecations and Removals

• Replacement: Use CustomJudge with prompt_template (Jinja syntax) and output_fields for structured evaluation results
• Note: DSPy-style prompt specs remain available in Traditional Monitoring but are no longer the recommended approach
• Action Required: If using custom LLM-as-a-Judge evaluators, adopt CustomJudge with Prompt Template syntax

• Security Updates: Applied routine security patches and version updates to application frameworks to stay up-to-date with the latest improvements.

​

What’s New and Improved

• GPU Acceleration: Sentiment evaluator now automatically detects and utilizes GPU resources when available
• Benefit: Dramatically faster evaluation performance - under 80ms on the scoring endpoint for all input sizes
• Note: Falls back to CPU processing if GPU is unavailable, maintaining backward compatibility
• Performance Optimization: Replaced model weights with GPU-compatible version
• Note: Input is truncated at 512 tokens as before

• Benefit: Streamlined cleanup of test and deprecated applications

​

Deprecations and Removals

• Removed from Available Evaluators: Answer Relevance is no longer available for new evaluation rules
• Note: Existing Answer Relevance rules will continue to function normally
• Replacement Timeline: Answer Relevance v2 will be introduced in Fiddler v26.3
• Benefit: The updated evaluator will provide improved accuracy and performance

• Security Updates: Applied routine security patches and version updates to application frameworks to stay up-to-date with the latest improvements.

​

What’s New and Improved

• Optimized Metric Fetching: Metric fetching engine now retrieves sparse data from the database more efficiently
• Benefit: Significantly reduced data transfer and processing overhead for metric queries
• Smart In-Memory Processing: Missing values are efficiently filled in-memory only when necessary
• Note: For example, when preserving zero-traffic bins
• Benefit: Improved query performance without sacrificing data completeness
• LRU Metadata Caching: Implemented Least Recently Used (LRU) caching for frequently accessed metadata
• Scope: Caches Projects, Models, Baselines, Segments, and Applications metadata
• Benefit: Further reduces database load and improves response times for common queries

• Span Latency Charts: Added new charts specifically designed for span latency visualization
• Benefit: Better visibility into performance characteristics of individual operations
• Granular Latency Filtering: Filter latencies by different time granularities
• Options: Microseconds, milliseconds, or seconds
• Benefit: Analyze performance at the appropriate scale for your use case

• Visual Threshold Tuning: Charts preview shows data points that fall under Critical and Warning thresholds
• Benefit: Dynamically fine-tune alert thresholds based on observed trends
• Benefit: Reduce false positives by visualizing actual data distribution before setting alerts

• Security Updates: Applied routine security patches and version updates to application frameworks to stay up-to-date with the latest improvements.

​

What’s New and Improved

• Default Safety Evaluator: New applications automatically include FTL safety evaluator rule
• Benefit: Immediate safety monitoring for all LLM spans and outputs without manual configuration
• Expanded Dashboard Charts: Default dashboards now include 4 charts in addition to traffic monitoring and traffic metrics
• Safety Analysis: Monitor safety evaluator results for responsible AI deployment
• Token Consumption: Track LLM token usage for cost optimization
• Accurate Token Counting: SDK-based token counting replaces previous estimation method
• Benefit: Precise cost tracking and optimization insights for LLM operations
• Production Monitoring Parity: Metric cards gaining feature parity with monitoring charts
• New Support: Token counts and custom attributes now available in metric cards
• Benefit: Consistent monitoring capabilities across different visualization types

• Token Count Alerts: Set thresholds for LLM token consumption
• Benefit: Proactive cost management and budget protection for API usage
• Attribute-Based Alerts: Create alerts on numerical and categorical custom attributes
• Benefit: Monitor business-specific metrics and application-specific dimensions

• New columns will have null values for historical events
• Future events must include data for newly added columns

• Event Deletion Bug Fixes: Resolved issues with event deletion operations in the data pipeline
• Benefit: Reliable data cleanup and management for production systems
• Rate Limit Handling: Added HTTP 429 responses for rate limit scenarios
• Benefit: Clear feedback when API rate limits are exceeded, enabling proper retry logic

​

Client Version

​

What’s New and Improved

• 1-Hour Traffic Alerts: Monitor traffic patterns with hourly granularity to catch issues quickly
• Benefit: Detect traffic anomalies and potential problems within an hour of occurrence
• 1-Day Traffic Alerts: Track daily traffic patterns for broader trend analysis
• Benefit: Identify long-term traffic patterns and plan capacity accordingly
• Configurable Thresholds: Set custom alert thresholds based on your application needs
• Note: Available for both hourly and daily monitoring periods

• Span-Level Attributes: Add granular metadata to individual operations
• Format: fiddler.span.user.{key} for operation-specific attributes
• Flexible Metadata: Capture any custom data relevant to your monitoring needs
• Benefit: Filter, group, and analyze traces using your own business dimensions
• Action Required: SDK update required to use this feature

• Custom Evaluation Logic: Define specific rules and conditions for evaluating LLM outputs
• Flexible Conditions: Set up complex evaluation criteria based on your use case
• Benefit: Tailor evaluation to your specific quality and safety requirements

• Flexible Model Selection: Use any LLM as an evaluator for your applications
• Custom Prompts: Define evaluation criteria with your own prompting strategies
• Benefit: Create domain-specific evaluators that understand your unique requirements

• Updated Evaluation Logic: Refined prompts and evaluation criteria for better accuracy
• Improved Response Quality: More consistent and reliable evaluation results
• Note: Input and output formats have been updated for improved performance
• Compatibility: These evaluators were deactivated in 25.20 to prepare for these improvements

• Simplified Steps: Reduced complexity in creating new GenAI applications
• Improved Guidance: Better in-app instructions and tooltips
• Benefit: Get your agentic and LLM applications monitored faster with less configuration

• Reduced Memory Usage: Delete operations now process data in chunks to minimize memory consumption
• Better Scalability: Handle large-scale deletions without resource constraints
• Performance Impact: Significantly lower memory footprint for bulk delete operations

​

Deprecations

• Deprecated Features:
• Toxicity enrichment
• Custom LLM Classifier enrichment
• Replacement: Use the new Custom LLMaaJ or Pre-Built LLMaaJ evaluators
• Enhanced evaluation capabilities with LLM-based judges
• More flexible and accurate evaluation
• Migration Path: Configure equivalent evaluators using the new LLMaaJ framework
• Benefit: Modern evaluation infrastructure with better accuracy and flexibility

​

Fixes and Security Updates

• Security Updates: Applied routine security patches and version updates to application frameworks to stay up-to-date with the latest improvements.

​

Client Version

​

Documentation Updates

• OpenTelemetry Integration: New comprehensive documentation for OpenTelemetry integration
• Quick Start guide for custom agent frameworks
• Advanced patterns and production configurations
• OpenTelemetry Quick Start →
• User-Defined Attributes: Documentation for adding custom attributes to traces
• Session and span-level attribute patterns
• Best practices for custom metadata

​

What’s New and Improved

• Improved Cryptocurrency Handling: Reduced false positives when analyzing content that references cryptocurrency and related financial terms
• Benefit: More accurate safety assessments for fintech and Web3 applications without unnecessary flagging of legitimate cryptocurrency discussions
• New Roleplaying Detection: Added roleplaying label to identify content where the model is instructed to adopt a specific persona or character
• Impact: Enhanced ability to detect potential prompt injection attempts and inappropriate persona adoption
• Note: Users with existing FTL Safety enrichments that did not specify label subsets via the classifiers configuration will automatically see the new roleplaying label on newly ingested data

​

Deprecations

• Replacement: LLM-as-a-Judge with Prompt Specs provides enhanced capabilities for custom evaluations with greater control and flexibility
• Migration Path: Refer to the LLM-as-a-Judge documentation for implementation guidance and examples
• Benefit: LLM-as-a-Judge offers more sophisticated evaluation workflows, better prompt engineering capabilities, and improved consistency for custom classification tasks

​

Fixes and Security Updates

• Security Updates: Applied routine security patches and version updates to application frameworks to stay up-to-date with the latest improvements.

​

What’s New and Improved

• Extended Active Sessions: Users remain authenticated while actively using Fiddler, eliminating the previous 2-hour hard timeout that logged out active users
• Benefit: Uninterrupted workflow for active users while maintaining robust security controls
• Intelligent Idle Detection: Sessions automatically expire after 2 hours of inactivity
• Default Configuration: 2-hour idle timeout, 24-hour maximum session lifetime, 1-hour access token lifetime
• Multi-Tab Session Synchronization: Session state is synchronized across multiple browser tabs, providing a consistent experience
• Benefit: Seamless transitions between browser tabs without unexpected logouts
• Proactive Token Refresh: Access tokens are automatically refreshed in the background when actively using the application

• Streamlined Job Display: The Succeeded/Failed tab shows only the jobs that require attention or review
• Failed streaming jobs remain visible for 30 days
• All batch ingestion jobs (successful and failed) remain visible for 30 days
• Successful streaming jobs are removed immediately upon completion
• Benefit: Cleaner job history makes it easier to identify jobs that need attention
• Performance Improvements: Chunk-level data purging reduces internal metadata table sizes
• Benefit: Faster page load times, especially for high-volume environments
• Smart Data Retention: The system maintains a 30-day retention period for jobs that may need to be replayed or investigated while removing unnecessary successful streaming job entries

​

Fixes and Security Updates

• Security Updates: Applied routine security patches and version updates to application frameworks to stay up-to-date with the latest improvements.

​

Fixes and Security Updates

• Security Updates: Applied routine security patches and version updates to application frameworks to stay up-to-date with the latest improvements.

​

Fixes and Security Updates

• Security Updates: Applied routine security patches and version updates to application frameworks to stay up-to-date with the latest improvements.

​

What’s New and Improved

• Login Redirection
• Users are now automatically redirected to their originally requested URL after signing in. This enhancement improves the user experience by eliminating the need to manually navigate back to the intended page after authentication.
• Customizable Identity Provider Group Prefix
• Organizations can now configure a custom prefix for identity provider group mapping instead of the default fiddler_ prefix. This enhancement allows Fiddler to work with existing organizational naming conventions for Active Directory, LDAP, and SSO groups.

• fiddler_ORG_ADMIN
• fiddler_ORG_MEMBER
• fiddler_

​

What’s New and Improved

• Dedicated Management Console: Access authentication administration at https://authn-{your-instance}.cloud.fiddler.ai
• Enhanced SSO Integration: Improved support for Okta, Microsoft Entra ID, Google, and Ping Identity with standardized configuration
• Identity Provider Group Sync: Automatic mapping of external groups to Fiddler teams with customizable group prefixes
• Role-Based Administration: Granular admin roles including “Org Owner” and “Org User Manager” for delegated management
• Mixed Authentication Support: Simultaneous SSO and email-based authentication methods

​

Fixes and Security Updates

• Security Updates: Applied routine security patches and version updates to application frameworks to stay up-to-date with the latest improvements.

​

What’s New and Improved

• Comprehensive PII Detection: Automatically detects and flags PII leakage across 25+ categories including personal identifiers, financial data, government IDs, and digital identifiers
• Detailed Output: Returns detected PII spans with labels, confidence scores, and character offsets for precise identification and redaction
• Enterprise Ready: Optimized for low latency and high-scale deployment with consistent sub-second response times
• Compliance Support: Helps meet privacy regulations like GDPR, CCPA, and HIPAA by preventing PII exposure

​

Fixes and Security Updates

• Security Updates: Applied routine security patches and version updates to application frameworks to stay up-to-date with the latest improvements.

• Security Updates: Applied routine security patches and version updates to application frameworks to stay up-to-date with the latest improvements.
• Alerts UI Consistency: Fixed various display inconsistencies in the Alerts interface to improve user experience and visual consistency across the platform.

​

What’s New and Improved

• Concurrent Enrichment Processing
• Introduced parallel processing for independent enrichment operations, significantly improving performance when multiple enrichments are applied to your LLM data
• Performance Impact: Reduces overall enrichment processing time by up to 70% when using multiple enrichments (e.g., combining PII detection, toxicity analysis, and sentiment scoring)
• Benefit: Faster data processing enables near real-time monitoring for high-volume LLM applications, reducing the delay between data ingestion and actionable insights
• Note: This feature is currently in controlled rollout. Contact your customer success manager if you’d like early access

​

Fixes and Security Updates

• Security Updates: Applied routine security patches and version updates to application frameworks to stay up-to-date with the latest improvements.

• Security Updates: Applied routine security patches and version updates to application frameworks to stay up-to-date with the latest improvements.

​

What’s New and Improved

• Custom LLM Enrichment & Evaluations Enhancements
• Improved Determinism: Temperature parameter now defaults to 0 for classification use cases, ensuring more consistent and repeatable results
• Enhanced Flexibility: The Evaluations endpoint now accepts a temperature parameter, matching the configuration options available in Enrichment
• Benefit: Teams can achieve more predictable classification outputs while maintaining the flexibility to adjust temperature when needed for specific use cases
• Fiddler Fast Trust Faithfulness v2.4.1
• Performance Boost: Up to 150ms faster processing on longer input texts
• Benefit: Reduced latency for faithfulness checks on documents and extended conversations, enabling more responsive LLM monitoring at scale
• Impact: Particularly beneficial for applications processing lengthy contexts or high-volume document analysis workflows

​

What’s New and Improved

• Custom Webhook Support for Alert Notifications
• Extended webhook integration capabilities beyond Slack and Microsoft Teams to support any webhook provider
• Key Features:
• Create custom webhook configurations for any third-party service or internal system
• Configure custom webhooks with alert rules for flexible notification routing
• Maintain existing Slack and Microsoft Teams integrations alongside custom webhooks
• Benefit: Organizations can now integrate Fiddler alerts with their preferred communication platforms and incident management systems
• Enhanced Data Ingestion Performance with ClickHouse Optimization
• Performance Improvements include:
• End-to-end ingestion latency reduced by up to 10x for faster data processing
• Label update operations now complete significantly faster
• Event deletion performance dramatically improved
• Enhanced ClickHouse storage efficiency and query performance
• Benefit: Teams can now process and analyze data in near real-time, enabling faster decision-making and more responsive monitoring of production models

​

Fixes and Security Updates

• Security Updates: Applied routine security patches and version updates to application frameworks to stay up-to-date with the latest improvements.

​

What’s New and Improved

• Updated Fiddler Fast Trust Faithfulness Model
• Classification Improvements:
• Improved accuracy on Q&A and simple knowledge-retrieval tasks
• Enhanced accuracy on Q&A with longer contexts
• Improved accuracy on “off-label” tasks like JSON-to-Text and Dialogue/Chat exchanges
• Performance Boost:
• 15-20% faster processing on longer contexts
• Enhanced Fiddler Fast Trust Safety Model
• Increased Safety model context window from 4,000 to 800,000 tokens
• Benefit: Enables comprehensive safety analysis on much larger documents and conversations without truncation
• Multi-threading for Embedding Enrichment
• Implemented parallel processing for embedding generation
• Performance Impact: At least a 5x improvement in processing speed for embedding enrichments
• Scalability: Can achieve even greater performance with additional threads and resources
• Benefit: Significantly reduces processing time for high-volume LLM monitoring pipelines
• Microsoft Teams Webhook Integration
• Added native support for Microsoft Teams webhook notifications alongside existing Slack integration
• Benefit: Teams can now receive alert notifications directly in their Microsoft Teams channels
• How to use: Configure Microsoft Teams webhooks in the Webhook Integrations tab of the Settings page
• Impact: Streamlines communication workflows for organizations using Microsoft Teams as their primary collaboration platform

​

Fixes and Security Updates

• Security Updates: Applied routine security patches and version updates to application frameworks to stay up-to-date with the latest improvements.

• Security Updates: Applied routine security patches and version updates to application frameworks to stay up-to-date with the latest improvements.

• New Glossary Feature: We’ve expanded our Product Concepts guide with a comprehensive glossary. Each term now has its own dedicated page containing:
• Detailed explanations
• Implementation guidance
• Related resources and references

​

What’s New and Improved

• Custom LLM Enrichment: Leverage Llama3.1 8B to categorize input data using your own prompts and custom categories
• Benefit: Enables flexible classification tasks tailored to your specific business needs, going beyond pre-defined enrichment types

​

Fixes and Security Updates

• Security Updates: Applied routine security patches and version updates to application frameworks to stay up-to-date with the latest improvements.

​

Documentation Updates

• Removed Deployment Guide: The Deployment Guide is no longer relevant to Fiddler SaaS and managed SaaS offerings.

​

Improvements

• Baseline Name Length: Increased the maximum allowed characters from 30 to 256. This change enables more descriptive baseline names for complex projects with multiple models and datasets.
• Action Required: None. Existing baselines remain unchanged.
• Enhanced Job Error Messages: Error messages during metrics aggregation now specifically identify which step in the data ingestion process failed, helping you troubleshoot pipeline issues faster.
• Benefit: Reduces debugging time by pinpointing in which step jobs are failing.

​

Fixes and Security Updates

• Security Updates: Applied routine security patches to container images and application frameworks to address recent vulnerabilities.
• Homepage Cache Timestamp: Fixed an issue where cached dashboard data would display incorrect “Last Updated” timestamps, leading to confusion about data freshness.

​

Documentation Updates

• Streamlined Structure: Reorganized documentation with improved navigation paths between related topics.
• New Section: Consolidated technical guides and API references into the “Technical Reference” section.

​

What’s New and Improved

​

Now Available in Public Preview

• UI-based Model Onboarding with Draft Mode: Iteratively refine your model schema before publishing. Validate sample data, collaborate with your team, and deploy with confidence. This streamlines the model deployment process for faster time-to-value. See our new Model Guides for details and best practices.

​

Improvements

• Enhanced Charts Framework: Implemented significant improvements to our charting system, delivering more consistent rendering and reliable performance across all dashboards and visualizations.
• Fast Faithfulness Trust Model Enhancements: Improved classification accuracy overall by 23% and reduced the Q&A benchmark error rate by 36%.

​

Fixes and Security Updates

• Performance Optimization: Updated application framework components to improve validation speed during model onboarding and data publishing processes.
• Enhanced Error Handling: Redesigned validation messages during Baseline creation to provide more actionable and detailed troubleshooting guidance.
• Concurrency Improvements: Optimized metrics calculation when model edits trigger recalculations, reducing processing time while preventing disruption to production data pipelines.

​

Documentation Updates

• Streamlined Structure: Merged the former UI Guide into the Product Guide for a more intuitive navigation experience.
• Expanded Content: Added comprehensive data publishing guides with practical examples and best practices for various data types and formats.
• New UI Onboarding Guide: Published detailed documentation for the new UI-based model onboarding feature, including step-by-step instructions and best practices.

​

What’s New and Improved

• Faster processing times for streaming data
• Reduced latency for real-time monitoring applications
• No delays from large batch upload operations

​

What’s New and Improved

​

Client Version

​

What’s New and Improved

​

What’s New and Improved

• Easy to use: We designed the Model Onboarding UI to be user-friendly, making it simple and intuitive to onboard your models.
• More accessible: This new feature makes it easy to onboard your models even if you’re not a Python expert.

• Onboard models for different tasks: Currently, it supports three model task types: Not Set, Regression, and Binary/Multiclass Classification.
• Upload your data: Upload your sample data, and Fiddler will automatically try to understand its structure, saving you time from entering details manually.
• Review and edit: You can quickly review and edit the data structure (schema) inferred by Fiddler.
• Define targets: Specify the target variable that your model is designed to predict.
• Provide model details: Give your model a name and other important information.

​

What’s New and Improved

• Pause Alert Notifications
• Users can now pause and resume notifications for specific alert rules without interrupting their evaluation. This feature helps reduce alert fatigue and prioritize critical alerts.
• Highlights:
• New bell icon for toggling notification status directly from the alert rule table.
• Notifications can be paused or resumed with clear visual feedback and confirmation messages.
• Benefits:
• Reduced alert fatigue.
• Continued evaluation of paused alerts without disruption.
• Model Schema Editing (Private Preview)
• Say goodbye to re-onboarding models for schema updates! This feature allows users to edit numerical and categorical columns and add metadata columns directly within the platform.
• Highlights:
• Adjust numerical ranges and add new categories to categorical columns.
• Add metadata columns for enhanced schema flexibility.
• Automatic recalculation of metrics and aggregates after edits.
• Benefits:
• Faster schema updates with no re-onboarding required.
• Keeps metrics and alerts in sync with the latest schema changes.
• UI-Based Model Onboarding (Private Preview)
• Simplify model onboarding with our new interactive UI. Add models without relying on Python APIs, making onboarding faster and more accessible to all team members.
• Highlights:
• Supports key task types: Not Set, Regression, and Binary Classification.
• Automatic schema inference and validation with error detection.
• Benefits:
• Streamlines onboarding for non-technical users.
• Reduces errors with built-in validation checks.

• The Model Schema Editing and UI-Based Model Onboarding features are available for private preview. For more details on how to participate, please refer to the private preview guide.

​

What’s New and Improved

• Native Integration with AWS SageMaker AI
• Fiddler is now natively supported within the newly launched Amazon SageMaker partner AI ecosystem. This integration enables enterprises to validate, monitor, analyze, and improve their ML models in production, all within their existing private and secure Amazon SageMaker AI environment. Read the official announcement here. Note Fiddler Python client version 3.7+ is required for this feature.
• Download Dataset Code in UI
• Now you can download your baseline and non-production datasets faster than ever with just a click! Building on the popular feature we introduced for production data in the Root Cause Analysis Events table, we’ve added ready-to-use Python code snippets right in the interface. Simply copy and paste these snippets to jumpstart your data analysis in your notebooks.
• Python Client Highlights
• The latest release of Fiddler’s Python client brings two powerful new convenience features to streamline your workflow:
• Our new Project.get_or_create() class function simplifies project creation in notebooks. This feature prevents name conflict errors during project creation when your notebook runs multiple times, saving you time and reducing the need for additional exception handling.
• We’ve also added model.remove_column(), a simpler way to remove columns during model onboarding. This function replaces the multi-step process previously required, making model configuration faster and more intuitive.
• To enhance reliability, we’ve implemented a configurable HTTP retry mechanism that you can fine-tune to match your network environment.
• For more details, please refer to the Python Client Release notes.

​

Discontinued

• The SQL Analyze Page Discontinued
• The legacy SQL Analyze page has been removed as of 24.18. The new Analyze experience within monitoring charts Root Cause Analysis now enables data table generation using Fiddler Query Language (FQL) and supports the creation of analytical charts such as confusion matrices, feature distribution charts, and more.

​

Client Version

​

What’s New and Improved

• Feature Analytics in Root Cause Analysis (Public Preview)
• The root cause analysis experience within monitoring charts now allows users to view feature distribution, feature correlation, and correlation matrix.

​

Discontinued

• SQL Methods in the Python Client Discontinued
• From Client 3.6 and onwards, get_slice and download_slice are discontinued. In their stead, use the new download_data method to download production and non-production data from your Fiddler models. If you have any questions or need any assistance migrating scripts using the deprecated methods, please contact your Fiddler customer success manager.
• Use of or support for Python 3.8 is discontinued by Fiddler. Note Python 3.8 has been designated End of Life as of October 7, 2024.

​

What’s New and Improved

• New Chart Type: Correlation Matrix (Public Preview)
• The Correlation Matrix chart enables users to visualize relationships between up to eight columns in a heatmap, making it easy to spot significant patterns. By clicking on any cell representing the relationship between two features, users can open a Feature Correlation chart for that pair, offering more detailed insights into the correlation score.
• Events Table in Root Cause Analysis (Public Preview)
• The root cause analysis experience within monitoring charts now allows users to perform deeper investigations by viewing and downloading up to 1,000 raw events, providing valuable insights for understanding and addressing potential issues.

​

What’s New and Improved

• New Chart Type: Metric Card (Public Preview)
• We’re excited to introduce the Metric Card chart type, which allows users to display up to four key numerical values in a clear and concise card format. This new visualization enhances data presentation by enabling quick insights into critical metrics, making it easier for decision-makers to spot trends or performance indicators at a glance.
• New Chart Type: Feature Correlation (Public Preview)
• The Feature Correlation chart, part of Feature Analytics charts, enables users to analyze and visualize the relationships between different features within their models. By offering a clear view of correlations, this tool supports more informed model diagnostics and refinement.

​

What’s New and Improved

• This release focused on system performance, stability, and security enhancements. These improvements ensure a smoother user experience and provide a more robust platform for future developments.

​

What’s New and Improved

• NEW Standalone Feature Distribution Chart (Public Preview)
• Create feature distribution charts for numerical and categorical data types that can be added to dashboards.
• Embedding Visualization UX Improvements
• User interface and usability improvements to the UMAP embedding visualization chart.
• Additional database performance improvements.

​

Deprecated and Decommissioned

• Fairness was decommissioned in v24.8, and the documentation has now been removed.

• Visualize charts to aid in analyzing model performance for Binary Classification, Multiclass Classification, and Regression models.
• Leverage applied segments in Performance Analytics charts to explore problematic cohorts of data.

​

What’s New and Improved

​

Release of Fiddler Platform Version 24.6

• Performance improvements
• Improved the performance of various modules / APIs.
• Improved observability which can help monitor health and performance of the operations.

​

Client Version

• Client version 3.1.2+ is required for the updates and features mentioned in this release.

​

Release of Fiddler Platform Version 24.5

• Support for model versions for streamlined model management

​

What’s New and Improved

• Model Versions
• Efficiently manage related models by creating structured versions, facilitating tasks like retraining and comparison analyses.
• Users can maintain model lineage, efficiently manage updates, flexibly modify schemas, and adjust parameters.
• Airgapped Enrichments (alpha)
• For privacy sensitive use cases, all data getting enriched stays within customer premises.
• New Deployment Base Images
• We have added new deployment base images to support model versioning.

​

Release of Fiddler Platform Version 24.4

• UMAP UI changes
• SSO integration changes
• New concept: Environments
• Fundamental changes to product concepts

​

What’s New and Improved

• UMAP UI
• Vertical scrolling instead of horizontal scrolling for data cards
• “View More” option to open data cards in maximized modal
• Ability to toggle between data cards in the maximized modal
• SSO integration changes
• Fiddler now integrates with Azure AD SSO, allowing you to leverage existing user roles for access control within Fiddler. This eliminates the need for manual user creation and simplifies user management within your organization.
• Environments
• Each Model now has two environments (Pre-Production and Production) used to house data in different ways.
• A Model’s Pre-Production environment is used to house non-time series data (Datasets).
• A Model’s Production environment is used to house time series data.
• Product concept changes
• Datasets are no longer stored at the Project level. Instead, they’re stored at the Model level under the Pre-Production Environment.
• The Model Details page has been updated with a new design.

​

Client 3.x Release

​

Deprecations and Removals

• All IDs will be UUIDs instead of strings.
• Dataset deletion is not allowed anymore.