Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minor: Add example for configuring SessionContext #12139

Merged
merged 1 commit into from
Aug 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 41 additions & 1 deletion datafusion/core/src/execution/context/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,9 @@ where
/// the state of the connection between a user and an instance of the
/// DataFusion engine.
///
/// See examples below for how to use the `SessionContext` to execute queries
/// and how to configure the session.
///
/// # Overview
///
/// [`SessionContext`] provides the following functionality:
Expand Down Expand Up @@ -200,7 +203,38 @@ where
/// # }
/// ```
///
/// # `SessionContext`, `SessionState`, and `TaskContext`
/// # Example: Configuring `SessionContext`
///
/// The `SessionContext` can be configured by creating a [`SessionState`] using
/// [`SessionStateBuilder`]:
///
/// ```
/// # use std::sync::Arc;
/// # use datafusion::prelude::*;
/// # use datafusion::execution::SessionStateBuilder;
/// # use datafusion_execution::runtime_env::{RuntimeConfig, RuntimeEnv};
/// // Configure a 4k batch size
/// let config = SessionConfig::new() .with_batch_size(4 * 1024);
///
/// // configure a memory limit of 1GB with 20% slop
/// let runtime_env = RuntimeEnv::new(
/// RuntimeConfig::new()
/// .with_memory_limit(1024 * 1024 * 1024, 0.80)
/// ).unwrap();
///
/// // Create a SessionState using the config and runtime_env
/// let state = SessionStateBuilder::new()
/// .with_config(config)
/// .with_runtime_env(Arc::new(runtime_env))
/// // include support for built in functions and configurations
/// .with_default_features()
/// .build();
///
/// // Create a SessionContext
/// let ctx = SessionContext::from(state);
/// ```
///
/// # Relationship between `SessionContext`, `SessionState`, and `TaskContext`
///
/// The state required to optimize, and evaluate queries is
/// broken into three levels to allow tailoring
Expand Down Expand Up @@ -1427,6 +1461,12 @@ impl From<&SessionContext> for TaskContext {
}
}

impl From<SessionState> for SessionContext {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes it possible to do

let ctx = SessionContext::from(state);

fn from(state: SessionState) -> Self {
Self::new_with_state(state)
}
}

/// A planner used to add extensions to DataFusion logical and physical plans.
#[async_trait]
pub trait QueryPlanner {
Expand Down
2 changes: 2 additions & 0 deletions datafusion/core/src/execution/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@

pub mod context;
pub mod session_state;
pub use session_state::{SessionState, SessionStateBuilder};
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this makes it possible to do

use datafusion::execution::SessionStateBuilder;


mod session_state_defaults;

pub use session_state_defaults::SessionStateDefaults;
Expand Down
Loading