Advanced Usage

A set of options is available in order to adapt the report generated.

General settings

Parameter

Type

Default

Description

title

string

Pandas Profiling Report

Title for the report, shown in the header and title bar.

pool_size

integer

0

Number of workers in thread pool. When set to zero, it is set to the number of CPUs available.

progress_bar

boolean

True

If True, pandas-profiling will display a progress bar.

The configuration can be changed in the following ways:

Configuration example
# Change the config when creating the report
profile = df.profile_report(title="Pandas Profiling Report", pool_size=1)

# Change the config after
profile.set_variable("html.minify_html", False)

profile.to_file("output.html")

Variable summary settings

Parameter

Type

Default

Description

sort

None, asc or desc

None

Sort the variables asc(ending), desc(ending) or None (leaves original sorting).

variables.descriptions

dict

{}

Ability to display a description alongside the descriptive statistics of each variable ({‘var_name’: ‘Description’}).

vars.num.quantiles

list[float]

[0.05,0.25,0.5,0.75,0.95]

The quantiles to calculate. Note that .25, .5 and .75 are required for other metrics median and IQR.

vars.num.skewness_threshold

integer

20

Warn if the skewness is above this threshold.

vars.num.low_categorical_threshold

integer

5

If the number of distinct values is smaller than this number, then the series is considered to be categorical. Set to 0 to disable.

vars.num.chi_squared_threshold

float

0.999

Set to zero to disable chi squared calculation.

vars.cat.length

boolean

True

Check the string length and aggregate values (min, max, mean, media).

vars.cat.characters

boolean

False

Check the distribution of characters and their Unicode properties. Often informative, but may be computationally expensive.

vars.cat.words

boolean

False

Check the distribution of words. Often informative, but may be computationally expensive.

vars.cat.cardinality_threshold

integer

50

Warn if the number of distinct values is above this threshold.

vars.cat.n_obs

integer

5

Display this number of observations.

vars.cat.chi_squared_threshold

float

0.999

Same as above.

vars.bool.n_obs

integer

3

Same as above.
Configuration example
profile = df.profile_report(
      sort='ascending',
      vars={
          'num':{'low_categorical_threshold': 0},
          'cat':{
            'length':True,
            'characters':False,
            'words':False,
            'n_obs': 5,
          }
      }
)

profile.set_variable('variables.descriptions',
    {
      'files': 'Files in the filesystem',
      'datec': 'Creation date',
      'datem': 'Modification date',
    }
)

profile.to_file("report.html")

Missing data overview plots

Parameter

Type

Default

Description

missing_diagrams.bar

boolean

True

Display a bar chart with counts of missing values for each column.

missing_diagrams.matrix

boolean

True

Display a matrix of missing values. Similar to the bar chart, but might provide overview of the co-occurrence of missing values in rows.

missing_diagrams.heatmap

boolean

True

Display a heatmap of missing values, that measures nullity correlation (i.e. how strongly the presence or absence of one variable affects the presence of another).

missing_diagrams.dendrogram

boolean

True

Display a dendrogram. Provides insight in the co-occurrence of missing values (i.e. columns that are both filled or both none).
Configuration example: disable heatmap and dendrogram for large datasets
profile = df.profile_report(
      missing_diagrams={
          'heatmap': False,
          'dendrogram': False,
      }
)
profile.to_file("report.html")

The missing data diagrams are generated by the missingno package.

Correlations

Parameter

Type

Default

Description

correlations.pearson.calculate

boolean

True

Whether to calculate this coefficient

correlations.pearson.warn_high_correlations

boolean

True

Warn for correlations higher than the threshold

correlations.pearson.threshold

float

0.9

Warning threshold

correlations.spearman.calculate

boolean

True

Whether to calculate this coefficient

correlations.spearman.warn_high_correlations

boolean

False

Warn for correlations higher than the threshold

correlations.spearman.threshold

float

0.9

Warning threshold

correlations.kendall.calculate

boolean

True

Whether to calculate this coefficient

correlations.kendall.warn_high_correlations

boolean

False

Warn for correlations higher than the threshold

correlations.kendall.threshold

float

0.9

Warning threshold

correlations.phi_k.calculate

boolean

True

Whether to calculate this coefficient

correlations.phi_k.warn_high_correlations

boolean

False

Warn for correlations higher than the threshold

correlations.phi_k.threshold

float

0.9

Warning threshold

correlations.cramers.calculate

boolean

True

Whether to calculate this coefficient

correlations.cramers.warn_high_correlations

boolean

True

Warn for correlations higher than the threshold

correlations.cramers.threshold

float

0.9

Warning threshold

Disable all correlations:

profile = df.profile_report(
     title="Report without correlations",
     correlations={
         "pearson": {"calculate": False},
         "spearman": {"calculate": False},
         "kendall": {"calculate": False},
         "phi_k": {"calculate": False},
         "cramers": {"calculate": False},
     },
 )

 # or using a shorthand that is available for correlations
    profile = df.profile_report(
     title="Report without correlations",
     correlations=None,
 )

Interactions

Parameter

Type

Default

Description

interactions.continuous

boolean

True

Generate a 2D scatter plot (or hexagonal binned plot) for all continuous variable pairs.

interactions.targets

list

[]

When a list of variable names is given, only interactions between these and all other variables are given.

The HTML Report

Parameter

Type

Default

Description

html.minify_html

bool

True

If True, the output html is minified using the htmlmin package.

html.use_local_assets

bool

True

If True, all assets (stylesheets, scripts, images) are stored locally. If False, a CDN is used for some stylesheets and scripts.

html.inline

boolean

True

If True, all assets are contained in the report. If False, then a web export is created, where all assets are stored in the ‘[REPORT_NAME]_assets/’ directory.

html.navbar_show

boolean

True

Whether to include a navigation bar in the report

html.style.theme

string

None

Select a ‘bootswatch’ theme. Available options: ‘flatly’ (dark) and ‘united’ (orange)

html.style.logo

string

A base64 encoded logo, to display in the navigation bar.

html.style.primary_color

string

#337ab7

The primary color to use in the report.

html.style.full_width

boolean

False

By default, the width of the report is fixed. If set to True, the full width of the screen is used.

Using a custom configuration file

To set the configuration of pandas-profiling using a custom file, you can start one of the sample configuration files below. Then, change the configuration to your liking.

from pandas_profiling import ProfileReport

profile = ProfileReport(df, config_file="your_config.yml")
profile.to_file("report.html")

Sample configuration files

A great way to get an overview of the possible configuration is to look through sample configuration files. The repository contains the following files:

Configuration shorthands

It’s possible to disable certain groups of features through configuration shorthands.

# Disable samples, correlations, missing diagrams and duplicates at once
r = ProfileReport(samples=None, correlations=None, missing_diagrams=None, duplicates=None, interactions=None)

# Or use the .set_variable method
r = ProfileReport()
r.set_variable("samples", None)
r.set_variable("duplicates", None)
r.set_variable("correlations", None)
r.set_variable("missing_diagrams", None)
r.set_variable("interactions", None)

Customise plots

A way how to pass arguments to the underlying matplotlib is to use the plot argument. It is possible to change the default format of images to png (default svg) using the key-pair image_format: "png" and also the resolution of the image using dpi: 800.

An example would be:

profile = ProfileReport(planets, title='Pandas Profiling Report', explorative=True,
                       plot={
                           'dpi':200,
                           'image_format': 'png'
                       })

Furthermore, it is possible to change the default values of histograms, the options for that are the following:

histogram:

x_axis_labels: True

# Number of bins (set to 0 to automatically detect the bin size)

bins: 50

# Maximum number of bins (when bins=0)

max_bins: 250

Customise correlation matrix

It’s possible to directly access the correlation matrix as well. That is done with the plot argument and then with the correlation key. It is possible to customise the palett, one can use the following list used in seaborn or create [their own custom matplotlib palette](https://matplotlib.org/stable/gallery/color/custom_cmap.html). Supported values are

` 'Accent', 'Accent_r', 'Blues', 'Blues_r', 'BrBG', 'BrBG_r', 'BuGn', 'BuGn_r', 'BuPu', 'BuPu_r', 'CMRmap', 'CMRmap_r', 'Dark2', 'Dark2_r', 'GnBu', 'GnBu_r', 'Greens', 'Greens_r', 'Greys', 'Greys_r', 'OrRd', 'OrRd_r', 'Oranges', 'Oranges_r', 'PRGn', 'PRGn_r', 'Paired', 'Paired_r', 'Pastel1', 'Pastel1_r', 'Pastel2', 'Pastel2_r', 'PiYG', 'PiYG_r', 'PuBu', 'PuBuGn', 'PuBuGn_r', 'PuBu_r', 'PuOr', 'PuOr_r', 'PuRd', 'PuRd_r', 'Purples', 'Purples_r', 'RdBu', 'RdBu_r', 'RdGy', 'RdGy_r', 'RdPu', 'RdPu_r', 'RdYlBu', 'RdYlBu_r', 'RdYlGn', 'RdYlGn_r', 'Reds', 'Reds_r', 'Set1', 'Set1_r', 'Set2', 'Set2_r', 'Set3', 'Set3_r', 'Spectral', 'Spectral_r', 'Wistia', 'Wistia_r', 'YlGn', 'YlGnBu', 'YlGnBu_r', 'YlGn_r', 'YlOrBr', 'YlOrBr_r', 'YlOrRd', 'YlOrRd_r', 'afmhot', 'afmhot_r', 'autumn', 'autumn_r', 'binary', 'binary_r', 'bone', 'bone_r', 'brg', 'brg_r', 'bwr', 'bwr_r', 'cividis', 'cividis_r', 'cool', 'cool_r', 'coolwarm', 'coolwarm_r', 'copper', 'copper_r', 'crest', 'crest_r', 'cubehelix', 'cubehelix_r', 'flag', 'flag_r', 'flare', 'flare_r', 'gist_earth', 'gist_earth_r', 'gist_gray', 'gist_gray_r', 'gist_heat', 'gist_heat_r', 'gist_ncar', 'gist_ncar_r', 'gist_rainbow', 'gist_rainbow_r', 'gist_stern', 'gist_stern_r', 'gist_yarg', 'gist_yarg_r', 'gnuplot', 'gnuplot2', 'gnuplot2_r', 'gnuplot_r', 'gray', 'gray_r', 'hot', 'hot_r', 'hsv', 'hsv_r', 'icefire', 'icefire_r', 'inferno', 'inferno_r', 'jet', 'jet_r', 'magma', 'magma_r', 'mako', 'mako_r', 'nipy_spectral', 'nipy_spectral_r', 'ocean', 'ocean_r', 'pink', 'pink_r', 'plasma', 'plasma_r', 'prism', 'prism_r', 'rainbow', 'rainbow_r', 'rocket', 'rocket_r', 'seismic', 'seismic_r', 'spring', 'spring_r', 'summer', 'summer_r', 'tab10', 'tab10_r', 'tab20', 'tab20_r', 'tab20b', 'tab20b_r', 'tab20c', 'tab20c_r', 'terrain', 'terrain_r', 'turbo', 'turbo_r', 'twilight', 'twilight_r', 'twilight_shifted', 'twilight_shifted_r', 'viridis', 'viridis_r', 'vlag', 'vlag_r', 'winter', 'winter_r' `

An example can be:

from pandas_profiling import ProfileReport

profile = ProfileReport(df, title='Pandas Profiling Report', explorative=True,
                     plot={
                         'correlation':{
                             'cmap': 'RdBu_r',
                             'bad': '#000000'}}
                     )

Similarly, one can change the palette for Missing values using the missing argument, eg:

from pandas_profiling import ProfileReport

profile = ProfileReport(df, title='Pandas Profiling Report', explorative=True,
                     plot={
                         'missing':{
                             'cmap': 'RdBu_r'}}
                     )