| Tag Databricks resources for cost attribution |
https://learn.microsoft.com/en-us/azure/databricks/admin/account-settings/usage-detail-tags |
| Use default Databricks compute policy families |
https://learn.microsoft.com/en-us/azure/databricks/admin/clusters/policy-families |
| Apply identity best practices in Azure Databricks |
https://learn.microsoft.com/en-us/azure/databricks/admin/users-groups/best-practices |
| Apply best practices for Databricks serverless workspaces |
https://learn.microsoft.com/en-us/azure/databricks/admin/workspace/serverless-workspaces-best-practices |
| Synthetically generate agent evaluation sets |
https://learn.microsoft.com/en-us/azure/databricks/agents/agent-evaluation/synthesize-evaluation-set |
| Load test Databricks Apps agents for QPS capacity |
https://learn.microsoft.com/en-us/azure/databricks/agents/custom-agents/load-test-agent-app |
| Measure RAG performance with retrieval and response metrics |
https://learn.microsoft.com/en-us/azure/databricks/agents/tutorials/ai-cookbook/evaluate-assess-performance |
| Define RAG application quality with evaluation sets |
https://learn.microsoft.com/en-us/azure/databricks/agents/tutorials/ai-cookbook/evaluate-define-quality |
| Evaluate and monitor RAG applications for quality, cost, latency |
https://learn.microsoft.com/en-us/azure/databricks/agents/tutorials/ai-cookbook/fundamentals-evaluation-monitoring-rag |
| Design and optimize RAG inference chains on Databricks |
https://learn.microsoft.com/en-us/azure/databricks/agents/tutorials/ai-cookbook/fundamentals-inference-chain-rag |
| Build Databricks unstructured RAG data pipelines |
https://learn.microsoft.com/en-us/azure/databricks/agents/tutorials/ai-cookbook/quality-data-pipeline-rag |
| Improve RAG application quality via key tuning knobs |
https://learn.microsoft.com/en-us/azure/databricks/agents/tutorials/ai-cookbook/quality-overview |
| Optimize RAG chain components for better responses |
https://learn.microsoft.com/en-us/azure/databricks/agents/tutorials/ai-cookbook/quality-rag-chain |
| Control coding agent costs with Unity Gateway budgets and routing |
https://learn.microsoft.com/en-us/azure/databricks/ai-gateway/coding-agent-costs |
| Apply performance best practices for AI Search |
https://learn.microsoft.com/en-us/azure/databricks/ai-search/best-practices |
| Load test Databricks AI Search endpoints |
https://learn.microsoft.com/en-us/azure/databricks/ai-search/endpoint-load-test |
| Improve Databricks AI Search retrieval quality |
https://learn.microsoft.com/en-us/azure/databricks/ai-search/retrieval-quality |
| Migrate Databricks library installs from init scripts |
https://learn.microsoft.com/en-us/azure/databricks/archive/compute/libraries-init-scripts |
| Apply best practices for Databricks compute policies |
https://learn.microsoft.com/en-us/azure/databricks/archive/compute/policies-best-practices |
| Use DBIO for transactional writes to cloud storage in Databricks |
https://learn.microsoft.com/en-us/azure/databricks/archive/legacy/dbio-commit |
| Optimize skewed joins in Databricks using skew hints |
https://learn.microsoft.com/en-us/azure/databricks/archive/legacy/skew-join |
| Migrate from Databricks Deep Learning Pipelines |
https://learn.microsoft.com/en-us/azure/databricks/archive/spark-3.x-migration/deep-learning-pipelines |
| Apply Azure Databricks platform administration best practices |
https://learn.microsoft.com/en-us/azure/databricks/cheat-sheet/administration |
| Optimize BI serving performance on Azure Databricks |
https://learn.microsoft.com/en-us/azure/databricks/cheat-sheet/bi-serving |
| Apply Databricks BI data preparation best practices |
https://learn.microsoft.com/en-us/azure/databricks/cheat-sheet/bi-serving-data-prep |
| Optimize Azure Databricks SQL warehouses for BI |
https://learn.microsoft.com/en-us/azure/databricks/cheat-sheet/bi-serving-sql-serving |
| Apply Azure Databricks compute creation best practices |
https://learn.microsoft.com/en-us/azure/databricks/cheat-sheet/compute |
| Implement Azure Databricks production job scheduling best practices |
https://learn.microsoft.com/en-us/azure/databricks/cheat-sheet/jobs |
| Apply Databricks-specific Power BI performance best practices |
https://learn.microsoft.com/en-us/azure/databricks/cheat-sheet/power-bi |
| Use AI-generated comments for Unity Catalog documentation |
https://learn.microsoft.com/en-us/azure/databricks/comments/ai-comments |
| Apply best practices for Databricks classic compute |
https://learn.microsoft.com/en-us/azure/databricks/compute/cluster-config-best-practices |
| Improve Databricks compute reliability with flexible node types |
https://learn.microsoft.com/en-us/azure/databricks/compute/flexible-node-types |
| Apply best practices for Databricks pools |
https://learn.microsoft.com/en-us/azure/databricks/compute/pool-best-practices |
| Apply best practices for Databricks serverless compute |
https://learn.microsoft.com/en-us/azure/databricks/compute/serverless/best-practices |
| Use Lakehouse Replay to validate runtime upgrades |
https://learn.microsoft.com/en-us/azure/databricks/compute/serverless/lakehouse-replay |
| Tune SQL warehouse settings for BI workloads |
https://learn.microsoft.com/en-us/azure/databricks/compute/sql-warehouse/bi-workload-settings |
| Control large interactive queries with Query Watchdog |
https://learn.microsoft.com/en-us/azure/databricks/compute/troubleshooting/query-watchdog |
| Configure dataset materialization for Databricks dashboards |
https://learn.microsoft.com/en-us/azure/databricks/dashboards/materialization |
| Apply Azure Databricks data engineering best practices |
https://learn.microsoft.com/en-us/azure/databricks/data-engineering/best-practices |
| Implement observability for Databricks streaming workloads |
https://learn.microsoft.com/en-us/azure/databricks/data-engineering/observability-best-practices |
| Best practices for Unity Catalog ABAC policies |
https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/abac/best-practices |
| Common ABAC row filtering and masking patterns |
https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/abac/common-patterns |
| Optimize ABAC row filter and mask performance |
https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/abac/performance |
| Apply Unity Catalog governance best practices |
https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/best-practices |
| Service policy examples for AI governance |
https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/service-policies/policy-examples |
| Work with legacy Hive metastore objects in Databricks |
https://learn.microsoft.com/en-us/azure/databricks/database-objects/hive-metastore |
| Safely use and migrate away from DBFS root |
https://learn.microsoft.com/en-us/azure/databricks/dbfs/dbfs-root |
| Apply best practices for DBFS and Unity Catalog |
https://learn.microsoft.com/en-us/azure/databricks/dbfs/unity-catalog |
| Apply Delta Lake best practices on Databricks |
https://learn.microsoft.com/en-us/azure/databricks/delta/best-practices |
| Handle Delta Lake limitations on S3 safely |
https://learn.microsoft.com/en-us/azure/databricks/delta/s3-limitations |
| Use selective overwrite options with Delta Lake on Databricks |
https://learn.microsoft.com/en-us/azure/databricks/delta/selective-overwrite |
| Apply recommended CI/CD workflows on Databricks |
https://learn.microsoft.com/en-us/azure/databricks/dev-tools/ci-cd/flows |
| View Databricks policy families via CLI |
https://learn.microsoft.com/en-us/azure/databricks/dev-tools/cli/reference/policy-families-commands |
| Develop Databricks Apps using supported frameworks and patterns |
https://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-apps/app-development |
| Apply best practices for Databricks Apps |
https://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-apps/best-practices |
| Add and use model serving endpoints in Databricks Apps |
https://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-apps/model-serving |
| Advanced configuration and usage of Databricks Connect |
https://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect/advanced |
| Test Databricks Connect Python code with pytest |
https://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect/python/testing |
| Handle async queries and interruptions with Databricks Connect |
https://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect/queries |
| Test Databricks Connect Scala code with ScalaTest |
https://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect/scala/testing |
| Call Databricks REST API with performance best practices |
https://learn.microsoft.com/en-us/azure/databricks/dev-tools/rest-api |
| Apply Databricks developer and CI/CD best practices |
https://learn.microsoft.com/en-us/azure/databricks/developers/best-practices |
| Choose between Unity Catalog volumes and workspace files |
https://learn.microsoft.com/en-us/azure/databricks/files/files-recommendations |
| Store and reference Databricks init scripts in workspace files |
https://learn.microsoft.com/en-us/azure/databricks/files/workspace-init-scripts |
| Curate effective Genie Agents for accurate answers |
https://learn.microsoft.com/en-us/azure/databricks/genie-agents/best-practices |
| Apply prompt and context best practices for Genie Code |
https://learn.microsoft.com/en-us/azure/databricks/genie-code/tips |
| Deep clone managed Iceberg tables in Unity Catalog |
https://learn.microsoft.com/en-us/azure/databricks/iceberg/clone |
| Apply Auto Loader best practices for reliable ingestion |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/cloud-object-storage/auto-loader/best-practices |
| Configure Azure Databricks Auto Loader for production |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/cloud-object-storage/auto-loader/production |
| Use automatic type widening in Databricks Auto Loader |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/cloud-object-storage/auto-loader/type-widening |
| Apply common COPY INTO data loading patterns |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/cloud-object-storage/copy-into/examples |
| Celigo connector FAQ and usage guidance |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/celigo-faq |
| Apply common patterns to Lakeflow ingestion pipelines |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/common-patterns |
| Apply Confluence connector behaviors and FAQs |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/confluence-faq |
| CrowdStrike Falcon Event Stream connector FAQ |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/crowdstrike-falcon-event-stream-faq |
| Apply Glean connector FAQs and usage guidance |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/glean-faq |
| Use Marketo connector effectively in Lakeflow Connect |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/marketo-faq |
| Apply Meta Ads connector usage guidance and FAQs |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/meta-ads-faq |
| MySQL Lakeflow connector FAQ and usage guidance |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/mysql-faq |
| Use Oracle integrated CDC connector FAQs and tips |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/oracle-faq |
| Apply PostgreSQL connector FAQs and guidance |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/postgresql-faq |
| Maintain and operate PostgreSQL Lakeflow pipelines |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/postgresql-maintenance |
| Use Productboard connector features and behaviors |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/productboard-faq |
| Reddit Ads connector usage FAQs and behaviors |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/reddit-ads-faq |
| Configure incremental ingestion of Salesforce formula fields |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/salesforce-formula-fields |
| Use Zerobus acknowledgment callbacks effectively |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/zerobus-callbacks |
| Choose Zerobus blocking methods for durability |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/zerobus-message-blocking |
| Implement resilient Zerobus recovery patterns |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/zerobus-recovery |
| Design Zerobus schemas for evolving data |
https://learn.microsoft.com/en-us/azure/databricks/ingestion/zerobus-schema-management |
| Use and configure Databricks cluster init scripts |
https://learn.microsoft.com/en-us/azure/databricks/init-scripts/ |
| Reference external files in Databricks init scripts |
https://learn.microsoft.com/en-us/azure/databricks/init-scripts/referencing-files |
| Test applications using the legacy Simba JDBC Driver |
https://learn.microsoft.com/en-us/azure/databricks/integrations/jdbc/testing |
| Test Databricks ODBC driver connections in code |
https://learn.microsoft.com/en-us/azure/databricks/integrations/odbc/testing |
| Schedule recurring SQL queries with backfill in Jobs |
https://learn.microsoft.com/en-us/azure/databricks/jobs/how-to/create-recurring-job |
| Apply classic compute best practices for Jobs |
https://learn.microsoft.com/en-us/azure/databricks/jobs/run-classic-jobs |
| Apply Databricks cost optimization best practices |
https://learn.microsoft.com/en-us/azure/databricks/lakehouse-architecture/cost-optimization/best-practices |
| Implement best practices for Databricks data and AI governance |
https://learn.microsoft.com/en-us/azure/databricks/lakehouse-architecture/data-governance/best-practices |
| Design observability and monitoring strategy for Databricks |
https://learn.microsoft.com/en-us/azure/databricks/lakehouse-architecture/deployment-guide/observability |
| Apply interoperability and usability best practices on Databricks |
https://learn.microsoft.com/en-us/azure/databricks/lakehouse-architecture/interoperability-and-usability/best-practices |
| Implement operational excellence best practices for Databricks |
https://learn.microsoft.com/en-us/azure/databricks/lakehouse-architecture/operational-excellence/best-practices |
| Implement performance efficiency best practices for Databricks |
https://learn.microsoft.com/en-us/azure/databricks/lakehouse-architecture/performance-efficiency/best-practices |
| Implement reliability best practices for Databricks workloads |
https://learn.microsoft.com/en-us/azure/databricks/lakehouse-architecture/reliability/best-practices |
| Apply security and compliance best practices in Azure Databricks |
https://learn.microsoft.com/en-us/azure/databricks/lakehouse-architecture/security-compliance-and-privacy/best-practices |
| Apply Databricks well-architected best practices across pillars |
https://learn.microsoft.com/en-us/azure/databricks/lakehouse-architecture/well-architected |
| Size and configure a Lakewatch detection warehouse |
https://learn.microsoft.com/en-us/azure/databricks/lakewatch/admin/detection-warehouse |
| Design effective Lakewatch detection rule SQL |
https://learn.microsoft.com/en-us/azure/databricks/lakewatch/detections/write-detection-rules |
| Run Lakewatch SIEM workflows end to end |
https://learn.microsoft.com/en-us/azure/databricks/lakewatch/quickstart-guide |
| Optimize pipeline clusters with enhanced autoscaling |
https://learn.microsoft.com/en-us/azure/databricks/ldp/auto-scaling |
| Apply Lakeflow pipeline best practices and patterns |
https://learn.microsoft.com/en-us/azure/databricks/ldp/best-practices/ |
| Ensure idempotent and exactly-once processing in Lakeflow |
https://learn.microsoft.com/en-us/azure/databricks/ldp/best-practices/processing-guarantees |
| Prepare Lakeflow pipelines for production readiness |
https://learn.microsoft.com/en-us/azure/databricks/ldp/best-practices/production-readiness |
| Implement AUTO CDC for SCD tables in Lakeflow |
https://learn.microsoft.com/en-us/azure/databricks/ldp/cdc |
| Handle environment version compatibility in Lakeflow pipelines |
https://learn.microsoft.com/en-us/azure/databricks/ldp/developer/environment-version-compatibility |
| Manage external Python dependencies in Lakeflow pipelines |
https://learn.microsoft.com/en-us/azure/databricks/ldp/developer/external-dependencies |
| Apply data quality expectations in pipelines |
https://learn.microsoft.com/en-us/azure/databricks/ldp/developer/ldp-python-ref-expectations |
| Apply advanced expectation patterns in pipelines |
https://learn.microsoft.com/en-us/azure/databricks/ldp/expectation-patterns |
| Use from_json schema inference in Lakeflow pipelines |
https://learn.microsoft.com/en-us/azure/databricks/ldp/from-json-schema-evolution |
| Perform full refreshes of streaming tables safely |
https://learn.microsoft.com/en-us/azure/databricks/ldp/full-refresh-st |
| Optimize stateful streaming with watermarks in pipelines |
https://learn.microsoft.com/en-us/azure/databricks/ldp/stateful-processing |
| Define transformations in Lakeflow pipelines |
https://learn.microsoft.com/en-us/azure/databricks/ldp/transform |
| Unit test Lakeflow pipeline transformations with mock data |
https://learn.microsoft.com/en-us/azure/databricks/ldp/unit-testing |
| Improve AI Runtime training performance and resiliency |
https://learn.microsoft.com/en-us/azure/databricks/machine-learning/ai-runtime/guides/performance-and-resiliency |
| Apply Hyperopt best practices and troubleshooting on Databricks |
https://learn.microsoft.com/en-us/azure/databricks/machine-learning/automl-hyperparam-tuning/hyperopt-best-practices |
| Improve Databricks AutoML forecasting with covariates |
https://learn.microsoft.com/en-us/azure/databricks/machine-learning/automl/automl-covariate-forecast |
| Implement point-in-time correct feature joins |
https://learn.microsoft.com/en-us/azure/databricks/machine-learning/feature-store/time-series |
| Benchmark Databricks LLM endpoints for latency and throughput |
https://learn.microsoft.com/en-us/azure/databricks/machine-learning/foundation-model-apis/prov-throughput-run-benchmark |
| Prepare large datasets for distributed training on Databricks |
https://learn.microsoft.com/en-us/azure/databricks/machine-learning/load-data/ddl-data |
| Configure load tests for custom model serving endpoints |
https://learn.microsoft.com/en-us/azure/databricks/machine-learning/model-serving/configure-load-test |
| Validate models before Databricks Model Serving deployment |
https://learn.microsoft.com/en-us/azure/databricks/machine-learning/model-serving/model-serving-pre-deployment-validation |
| Optimize Databricks Model Serving endpoints for production |
https://learn.microsoft.com/en-us/azure/databricks/machine-learning/model-serving/production-optimization |
| Load test Databricks model serving endpoints with Locust |
https://learn.microsoft.com/en-us/azure/databricks/machine-learning/model-serving/what-is-load-test |
| Tune and scale Ray clusters on Databricks |
https://learn.microsoft.com/en-us/azure/databricks/machine-learning/ray/scale-ray |
| Apply deep learning best practices on Azure Databricks |
https://learn.microsoft.com/en-us/azure/databricks/machine-learning/train-model/dl-best-practices |
| Adapt existing Apache Spark workloads to Databricks |
https://learn.microsoft.com/en-us/azure/databricks/migration/spark |
| Implement agent observability and quality on Databricks |
https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/agent-observability-and-quality |
| Align MLflow LLM judges with human feedback |
https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/eval-monitor/align-judges |
| Create guidelines-based LLM judges in MLflow |
https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/eval-monitor/concepts/judges/guidelines |
| Collect human feedback and build MLflow datasets |
https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/human-feedback/ |
| Enable experts to label MLflow traces with Review App |
https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/human-feedback/expert-feedback/label-existing-traces |
| Evaluate and compare MLflow prompt versions |
https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/prompt-version-mgmt/prompt-registry/evaluate-prompts |
| Enrich MLflow traces with tags, context, and feedback |
https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/tracing/enrich-traces |
| Use MLflow traces to observe and debug agents |
https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/tracing/observe-with-traces/ |
| Detect and monitor issues across MLflow traces |
https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/tracing/observe-with-traces/analyze-traces |
| Optimize multiple chained prompts with MLflow |
https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/tutorials/examples/multi-prompt-optimization |
| Apply software engineering practices to Databricks notebooks |
https://learn.microsoft.com/en-us/azure/databricks/notebooks/best-practices |
| Run Databricks notebooks safely and efficiently |
https://learn.microsoft.com/en-us/azure/databricks/notebooks/run-notebook |
| Test Databricks notebooks with built-in tools |
https://learn.microsoft.com/en-us/azure/databricks/notebooks/test-notebooks |
| Monitor active queries in Lakebase Postgres |
https://learn.microsoft.com/en-us/azure/databricks/oltp/projects/active-queries |
| Analyze Lakebase query performance history |
https://learn.microsoft.com/en-us/azure/databricks/oltp/projects/query-performance |
| Apply Azure Databricks performance optimization recommendations |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/ |
| Migrate from deprecated Bloom filter indexes on Databricks |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/bloom-filters |
| Use dynamic file pruning to optimize Delta joins |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/dynamic-file-pruning |
| Manage isolation levels and write conflicts for Delta tables |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/isolation/ |
| Use row-level concurrency to reduce Delta write conflicts |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/isolation/row-level-concurrency |
| Optimize Delta MERGE operations with low shuffle merge |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/low-shuffle-merge |
| Leverage predictive I/O optimizations with Photon |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/predictive-io |
| Use predictive optimization for managed tables |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/predictive-optimization |
| Optimize interval and range joins on Azure Databricks |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/range-join |
| Use Spark UI to optimize Databricks cost and performance |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/ |
| Analyze Spark jobs timeline for Databricks performance |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/jobs-timeline |
| Diagnose long Spark jobs using Databricks UI |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/long-spark-stage |
| Optimize high I/O Spark stages in Databricks |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/long-spark-stage-io |
| Diagnose skew and spill in Databricks Spark stages |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/long-spark-stage-page |
| Fix long Spark stages with one task in Databricks |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/one-spark-task |
| Debug slow low-I/O Spark stages in Databricks |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/slow-spark-stage-low-io |
| Optimize many small Spark jobs on Databricks |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/small-spark-jobs |
| Identify expensive reads in Databricks SQL DAG |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/spark-dag-expensive-read |
| Resolve overloaded Spark drivers on Databricks |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/spark-driver-overloaded |
| Fix gaps between Spark jobs in Databricks |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/spark-job-gaps |
| Detect excessive Delta rewrites in Databricks Spark |
https://learn.microsoft.com/en-us/azure/databricks/optimizations/spark-ui-guide/spark-rewriting-data |
| Apply best practices for Partner Connect setup |
https://learn.microsoft.com/en-us/azure/databricks/partner-connect/best-practice |
| Configure networking for Databricks Lakehouse Federation |
https://learn.microsoft.com/en-us/azure/databricks/query-federation/networking |
| Optimize performance of Lakehouse Federation queries |
https://learn.microsoft.com/en-us/azure/databricks/query-federation/performance-recommendations |
| Encrypt inter-node traffic for Databricks clusters |
https://learn.microsoft.com/en-us/azure/databricks/security/keys/encrypt-otw |
| Apply custom DNS best practices for Databricks VNets |
https://learn.microsoft.com/en-us/azure/databricks/security/network/classic/custom-dns |
| Use SIGNAL and RESIGNAL in Databricks SQL handlers |
https://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/control-flow/signal-stmt |
| Estimate distinct counts with approx_count_distinct |
https://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/functions/approx_count_distinct |
| Use session_user instead of deprecated current_user |
https://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/functions/current_user |
| Define liquid clustering with CLUSTER BY in Databricks |
https://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/sql-ref-syntax-ddl-cluster-by |
| Use OFFSET and LIMIT for pagination in Databricks SQL |
https://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/sql-ref-syntax-qry-select-offset |
| Use REPLACE WHERE flows for targeted recomputes |
https://learn.microsoft.com/en-us/azure/databricks/sql/mv-st/flows-replace-where |
| Author effective SQL patterns for Databricks alerts |
https://learn.microsoft.com/en-us/azure/databricks/sql/user/alerts/query-patterns |
| Apply Databricks SQL query performance insights |
https://learn.microsoft.com/en-us/azure/databricks/sql/user/queries/performance-insights |
| Optimize Databricks SQL with RELY constraints |
https://learn.microsoft.com/en-us/azure/databricks/sql/user/queries/query-optimization-constraints |
| Manage Structured Streaming checkpoints correctly |
https://learn.microsoft.com/en-us/azure/databricks/structured-streaming/checkpoints |
| Run multiple Structured Streaming queries per cluster |
https://learn.microsoft.com/en-us/azure/databricks/structured-streaming/multiple-streams |
| Run Structured Streaming workloads in production |
https://learn.microsoft.com/en-us/azure/databricks/structured-streaming/production |
| Use real-time mode for ultra-low latency streaming |
https://learn.microsoft.com/en-us/azure/databricks/structured-streaming/real-time/ |
| Optimize Databricks real-time streaming query performance |
https://learn.microsoft.com/en-us/azure/databricks/structured-streaming/real-time/performance |
| Use rolling windows for real-time aggregations in Databricks |
https://learn.microsoft.com/en-us/azure/databricks/structured-streaming/rolling-window |
| Optimize stateful Structured Streaming queries on Databricks |
https://learn.microsoft.com/en-us/azure/databricks/structured-streaming/stateful-streaming |
| Optimize stateless Structured Streaming queries |
https://learn.microsoft.com/en-us/azure/databricks/structured-streaming/stateless-streaming |
| Monitor Databricks Structured Streaming queries effectively |
https://learn.microsoft.com/en-us/azure/databricks/structured-streaming/stream-monitoring |
| Tune Structured Streaming trigger intervals on Databricks |
https://learn.microsoft.com/en-us/azure/databricks/structured-streaming/triggers |
| Apply watermarks for stateful streaming control |
https://learn.microsoft.com/en-us/azure/databricks/structured-streaming/watermarks |
| Leverage data skipping for faster queries |
https://learn.microsoft.com/en-us/azure/databricks/tables/data-skipping |
| Optimize partition discovery for external tables |
https://learn.microsoft.com/en-us/azure/databricks/tables/external-partition-discovery |
| Optimize VARIANT performance with shredding |
https://learn.microsoft.com/en-us/azure/databricks/tables/features/variant-shredding |
| Use table history and time travel safely |
https://learn.microsoft.com/en-us/azure/databricks/tables/history |
| Enrich Databricks tables with comments and metadata |
https://learn.microsoft.com/en-us/azure/databricks/tables/operations/custom-metadata |
| Safely drop or replace Databricks tables by type |
https://learn.microsoft.com/en-us/azure/databricks/tables/operations/drop-table |
| Optimize Delta and Iceberg table file layout |
https://learn.microsoft.com/en-us/azure/databricks/tables/operations/optimize |
| Use VACUUM to reclaim storage and ensure compliance |
https://learn.microsoft.com/en-us/azure/databricks/tables/operations/vacuum |
| Interpret table size and reclaim storage in Databricks |
https://learn.microsoft.com/en-us/azure/databricks/tables/size |
| Control Delta and Iceberg data file sizes |
https://learn.microsoft.com/en-us/azure/databricks/tables/tune-file-size |
| Safely evolve Delta and Iceberg table schemas |
https://learn.microsoft.com/en-us/azure/databricks/tables/update-schema |
| Aggregate data using batch, materialized views, and streaming |
https://learn.microsoft.com/en-us/azure/databricks/transform/aggregation |
| Design Delta Lake data models on Databricks |
https://learn.microsoft.com/en-us/azure/databricks/transform/data-modeling |
| Implement joins for batch and streaming in Databricks |
https://learn.microsoft.com/en-us/azure/databricks/transform/join |
| Optimize join performance on Azure Databricks |
https://learn.microsoft.com/en-us/azure/databricks/transform/optimize-joins |
| Clean and validate data on Azure Databricks |
https://learn.microsoft.com/en-us/azure/databricks/transform/validate |
| Implement session-scoped Scala and Java UDFs |
https://learn.microsoft.com/en-us/azure/databricks/udf/scala |
| Access task context inside Databricks UDFs |
https://learn.microsoft.com/en-us/azure/databricks/udf/udf-task-context |
| Download internet data into Azure Databricks volumes |
https://learn.microsoft.com/en-us/azure/databricks/volumes/download-internet-files |
