Datalore : Supercharge DataOps with Enhanced Security, AI BYOK & New Explorer Cells

Introduction

In the rapidly evolving landscape of data science and MLOps, the demand for secure, efficient, and collaborative environments is paramount. Data professionals constantly seek tools that not only accelerate insights but also integrate seamlessly with enterprise-grade security and operational protocols. JetBrains' Datalore has consistently aimed to meet these needs, providing a powerful online notebook environment. With the first major release of 2026, Datalore 2026.1 introduces a suite of features designed to elevate the data exploration experience, fortify security postures, and empower organizations with greater control over their AI deployments.

This update addresses critical pain points, from streamlining initial data investigation to ensuring robust data governance for AI models and hardening on-premises deployments through advanced Kubernetes patterns. These enhancements are already live for Datalore Cloud users and are available for Datalore On-Premises administrators via a simple update.

Core Concepts

Datalore 2026.1 brings three significant advancements, each addressing distinct facets of the data workflow:

New Data Explorer Cells: Interactive Insights at Your Fingertips

The traditional notebook experience often requires switching between code and external tools for comprehensive data profiling and visualization. Data Explorer Cells integrate these capabilities directly into your Datalore notebook. These specialized cells allow users to interactively analyze dataframes, generate quick visualizations, and understand data distributions without writing extensive plotting code. They act as a dynamic scratchpad for initial data assessment, helping data scientists quickly grasp dataset characteristics and identify potential issues or trends before diving into complex modeling.

Instance-Wide BYOK for AI: Unlocking Data Sovereignty

Bring Your Own Key (BYOK) is a critical security feature for enterprises handling sensitive data. Datalore 2026.1 extends BYOK support to AI artifacts across an entire instance. This means organizations can use their own customer-managed encryption keys (CMEK) from a Key Management Service (KMS) to encrypt not just data stored in Datalore, but also AI model weights, training data, and inference logs. This provides an additional layer of security and helps meet stringent compliance requirements, giving customers complete control over the encryption lifecycle of their most valuable AI assets.

Stronger Security via Sidecar Containers in Kubernetes: Hardening On-Premises

For Datalore On-Premises deployments running on Kubernetes, this update introduces enhanced security through the strategic use of sidecar containers. Sidecar containers are a popular pattern in Kubernetes to augment the functionality of a primary application container without modifying its core image. In Datalore 2026.1, security sidecars can be configured to enforce network policies, perform real-time auditing, manage secret injection, or even provide secure proxying for external services. This modular approach allows administrators to implement granular security controls and monitoring capabilities directly alongside their Datalore components, bolstering the overall integrity and compliance of the platform.

Implementation Guide

Activating New Data Explorer Cells

Utilizing Data Explorer Cells is intuitive and requires no complex setup.

  1. Open a Datalore Notebook: Navigate to an existing notebook or create a new one.
  2. Add a New Cell: Click the '+' icon or use the shortcut to add a new cell.
  3. Select "Data Explorer": From the cell type dropdown, choose "Data Explorer."
  4. Attach Data: Connect the cell to a DataFrame variable in your current notebook session or upload a new dataset.
  5. Explore: Use the interactive UI to filter, sort, visualize distributions, and generate quick plots.
# Example: Load a DataFrame and make it available for Data Explorer Cell
import pandas as pd
df = pd.read_csv('your_dataset.csv')
# The Data Explorer Cell will automatically detect 'df' in scope.

Configuring Instance-Wide BYOK for AI

This configuration is typically handled by Datalore instance administrators.

  1. Prepare Your KMS Key: Ensure you have a customer-managed encryption key (CMEK) set up in your chosen Key Management Service (e.g., AWS KMS, Azure Key Vault, Google Cloud KMS).
  2. Obtain Key Identifier: Get the ARN or equivalent identifier for your KMS key.
  3. Access Datalore Admin Panel: Log in to your Datalore instance as an administrator.
  4. Navigate to Security Settings: Find the "Security & Compliance" section.
  5. Configure BYOK for AI:

    Input your KMS key identifier in the designated field. You may also need to provide IAM roles or service accounts that Datalore will use to access the key.

    # Placeholder for Datalore Admin API/UI configuration
{
  "byok": {
    "enabled": true,
    "kmsProvider": "AWS_KMS", // or AZURE_KEY_VAULT, GCP_KMS
    "keyIdentifier": "arn:aws:kms:region:account-id:key/your-cmek-id",
    "accessRoleArn": "arn:aws:iam::account-id:role/datalore-kms-access"
  }
}
  6. Apply and Restart: Apply the changes, which may require a restart of specific Datalore services for the new encryption policies to take effect.

Implementing Sidecar Containers for Kubernetes Security (On-Premises)

This requires modifying your Datalore Kubernetes deployment manifests.

  1. Identify Target Deployments: Determine which Datalore components (e.g., notebook servers, data proxies) will benefit from sidecar-based security.
  2. Define Sidecar Container: Create a sidecar container definition within the pod's spec.containers. This could be for a network policy agent, a secret injector, or a logging proxy.
  3. Configure Sidecar Logic: Implement the specific security logic within the sidecar. For example, a network policy sidecar might use iptables or interact with a CNI plugin.
  4. Update Datalore K8s Manifests: Apply these sidecar definitions to your Datalore deployment YAMLs.
# Example: Adding a security sidecar to a Datalore notebook server pod
apiVersion: apps/v1
kind: Deployment
metadata:
  name: datalore-notebook-server
spec:
  template:
    spec:
      containers:
      - name: notebook-container
        image: jetbrains/datalore-notebook:2026.1
        # ... existing notebook container config ...
      - name: security-sidecar
        image: your-security-sidecar-image:1.0 # e.g., a network policy enforcer
        ports:
        - containerPort: 8080 # if acting as a proxy
        env:
        - name: SECURITY_POLICY_CONFIG
          value: "/etc/security/policy.json"
        volumeMounts:
        - name: policy-config
          mountPath: "/etc/security"
      volumes:
      - name: policy-config
        configMap:
          name: datalore-security-policy
  1. Apply Changes: Use kubectl apply -f your-datalore-deployment.yaml to update your Kubernetes deployments.

Automating This in CI/CD

Integrating these features into your CI/CD pipelines ensures consistency and reduces manual errors, especially for on-premises deployments.

BYOK Configuration Automation

For Datalore On-Premises, managing BYOK settings can be automated using infrastructure-as-code tools like Terraform or Ansible. Your CI/CD pipeline (e.g., GitHub Actions, Jenkins) can:

  1. Terraform/CloudFormation: Provision/update KMS keys and associated IAM roles.
  2. Datalore API Integration: Utilize Datalore's administrative API (if available for BYOK config) to automatically update the byok settings in Datalore after key creation.
  3. Secret Management: Securely pass key identifiers and access credentials from a vault (e.g., HashiCorp Vault) to your automation scripts.
# GitHub Actions workflow snippet for BYOK config
name: Update Datalore BYOK Config
on:
  push:
    branches:
      - main
    paths:
      - 'datalore/byok-config.json'

jobs:
  update-byok:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - name: Configure AWS Credentials
      uses: aws-actions/configure-aws-credentials@v1
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws-region: us-east-1
    - name: Update Datalore BYOK
      run: |
        # Placeholder: Use Datalore's admin CLI or API to apply config
        # datalore-admin config set byok-ai --file datalore/byok-config.json
        echo "Datalore BYOK configuration updated successfully."

Sidecar Security Deployment Automation

Automating sidecar deployments is crucial for maintaining consistent security posture across all Datalore components on Kubernetes.

  1. Version Control Manifests: Store your Datalore Kubernetes manifests, including sidecar definitions, in a Git repository.
  2. Helm/Kustomize: Use templating tools like Helm or Kustomize to manage and parameterize sidecar configurations.
  3. GitOps Workflows: Implement a GitOps approach (e.g., using Argo CD or Flux) where changes to your Git repository automatically trigger Kubernetes deployments, ensuring the correct sidecars are always running.
# Jenkins Pipeline snippet for K8s deployment with sidecars
pipeline {
    agent any
    stages {
        stage('Checkout Code') {
            steps {
                git url: 'https://github.com/your-org/datalore-k8s-config.git'
            }
        }
        stage('Deploy Datalore with Sidecars') {
            steps {
                script {
                    sh 'kubectl apply -f datalore/deployments/with-sidecars/'
                    echo 'Datalore deployments with security sidecars updated.'
                }
            }
        }
    }
}

Comparison vs Alternatives

While many platforms offer interactive notebooks (JupyterLab, Databricks, Google Colab) and cloud services provide BYOK, Datalore 2026.1 strengthens its position by offering a cohesive, enterprise-focused solution:

  • Data Explorer Cells: While other platforms have varying levels of integrated data profiling, Datalore's new cells provide a seamless, no-code/low-code interactive experience directly within the notebook, reducing context switching often seen with external profiling tools.
  • Instance-Wide BYOK for AI: Many cloud providers offer BYOK for storage, but instance-wide application to all AI artifacts within a managed notebook environment like Datalore is a significant differentiator for stringent compliance and data sovereignty requirements, especially compared to more open-source or consumer-grade notebook solutions.
  • Kubernetes Sidecar Security: This feature directly addresses the needs of enterprise on-premises deployments. While competitors like Databricks offer managed security, Datalore's sidecar approach empowers on-prem users with granular control and the flexibility to integrate custom security measures within their existing Kubernetes infrastructure, a level of control often harder to achieve with purely managed services.

Datalore's integrated approach across cloud and on-premises environments, combined with these specialized security and data exploration features, positions it as a robust choice for organizations prioritizing both productivity and compliance.

Best Practices

  • Data Explorer Cells: Use them for initial data sanity checks, rapid hypothesis testing, and creating shareable, interactive summaries before deep coding. Don't replace full analysis with them, but leverage their speed for exploration.
  • BYOK for AI: Implement a strict key rotation policy. Monitor KMS key access logs for any anomalies. Ensure appropriate IAM roles or service accounts are configured with the principle of least privilege for Datalore to access the keys. Regularly audit your encryption status.
  • Sidecar Containers in Kubernetes: Define clear responsibilities for each sidecar. Monitor sidecar health and logs closely. Start with simpler sidecar patterns (e.g., logging, network policy enforcement) and gradually introduce more complex ones. Regularly review your sidecar configurations against updated security standards and vulnerabilities.
  • CI/CD Integration: Treat all configuration (BYOK settings, Kubernetes manifests) as code. Implement pull request reviews for all changes. Use automated testing to validate deployments and configurations after CI/CD runs.

Conclusion

Datalore 2026.1 represents a significant leap forward in empowering data professionals and reinforcing enterprise security. The introduction of Data Explorer Cells streamlines the path from raw data to actionable insights, making initial exploration faster and more intuitive. Instance-Wide BYOK for AI models provides unprecedented control over sensitive AI assets, meeting the most demanding compliance standards. Furthermore, the enhanced security via sidecar containers in Kubernetes offers on-premises users powerful tools to harden their deployments. These features collectively underscore Datalore's commitment to delivering a secure, performant, and user-friendly environment for the future of data science and MLOps.

It's time to explore, secure, and innovate with Datalore 2026.1.

Comments

Post a Comment

Popular posts from this blog

Real-world Terraform scenarios to test and improve your Infrastructure as Code skills

Terraform Scenario Questions 15 min read Introduction Terraform is a powerful Infrastructure as Code tool that presents various challenges in real-world scenarios. These scenario-based questions cover common issues and advanced use cases that DevOps engineers encounter when managing cloud infrastructure with Terraform. Scenario 1: Multi-Environment State Management Situation Your organization has development, staging, and production environments. Currently, all environments share the same Terraform state file, causing conflicts and accidental production changes. Current Problem # Current problematic structure terraform/ ├── main.tf ├── variables.tf ├── terraform.tfvars └── terraform.tfstate # Shared across all environments Solution: Workspace-based Approach # Directory structure terraform/ ...
Read more

Azure Kubernetes Service (AKS) Complete Guide

Azure Kubernetes Service (AKS) Complete DevOps Guide Table of Contents Introduction to AKS AKS Architecture Cluster Creation & Management Networking Configuration Storage Management Security Best Practices Scaling Strategies CI/CD Integration Monitoring & Logging GitOps with AKS Disaster Recovery Cost Optimization Best Practices Introduction to Azure Kubernetes Service (AKS) Azure Kubernetes Service (AKS) is Microsoft's managed Kubernetes offering that simplifies deploying, managing, and scaling containerized applications using Kubernetes. It handles critical tasks like health monitoring, maintenance, and security patching, allowing DevOps teams to focus on applications rather than infrastru...
Read more

Automate Your DevOps Documentation: `iac-to-docs` Lands on PyPI with AI Power

Introduction Core Concepts: What is `iac-to-docs`? Implementation Guide: Getting Started Automating Documentation in CI/CD Comparison vs. Alternatives Best Practices for AI-Generated Docs Conclusion Introduction In the fast-paced world of DevOps, infrastructure-as-code (IaC) has become the bedrock of reliable and scalable systems. Yet, while our infrastructure is codified, our documentation often remains a manual, laborious, and frequently outdated chore. The gap between deployed infrastructure and its corresponding documentation grows with every commit, leading to onboarding challenges, operational silos, and increased incident resolution times. Historically, teams have grappled with this "documentation debt" through manual updates, custom scripts, or simply by accepting the risk. We're thrilled to announce a significant leap forward in solving this persistent problem: the open-source AI agent,...
Read more