View a markdown version of this page

Updating Debugger system monitoring and framework profiling configuration while a training job is running - Amazon SageMaker AI

Updating Debugger system monitoring and framework profiling configuration while a training job is running

Note

On 6/30/27, AWS will discontinue support for Amazon SageMaker Profiler. After 6/30/27, you will no longer be able to access the Profiler console or Profiler resources. For more information, see Profiler availability change.

If you want to activate or update the Debugger monitoring configuration for a training job that is currently running, use the following SageMaker AI estimator extension methods:

  • To activate Debugger system monitoring for a running training job and receive a Debugger profiling report, use the following:

    estimator.enable_default_profiling()

    When you use the enable_default_profiling method, Debugger initiates the default system monitoring and the ProfileReport built-in rule, which generates a comprehensive profiling report at the end of the training job. This method can be called only if the current training job is running without both Debugger monitoring and profiling.

    For more information, see estimator.enable_default_profiling in the Amazon SageMaker Python SDK.

  • To update system monitoring configuration, use the following:

    estimator.update_profiler(
    system_monitor_interval_millis=500
)

    For more information, see estimator.update_profiler in the Amazon SageMaker Python SDK.