{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 深度概率编程库\n", "\n", "[![下载Notebook](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/resource/_static/logo_notebook.svg)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/master/probability/zh_cn/mindspore_probability.ipynb) [![下载样例代码](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/resource/_static/logo_download_code.svg)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/master/probability/zh_cn/mindspore_probability.py) [![查看源文件](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/resource/_static/logo_source.svg)](https://gitee.com/mindspore/docs/blob/master/docs/probability/docs/source_zh_cn/probability.ipynb)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "MindSpore深度概率编程的目标是将深度学习和贝叶斯学习结合,包括概率分布、概率分布映射、深度概率网络、概率推断算法、贝叶斯层、贝叶斯转换和贝叶斯工具箱,面向不同的开发者。对于专业的贝叶斯学习用户,提供概率采样、推理算法和模型构建库;另一方面,为不熟悉贝叶斯深度学习的用户提供了高级的API,从而不用更改深度学习编程逻辑,即可利用贝叶斯模型。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 概率分布\n", "\n", "概率分布(`mindspore.nn.probability.distribution`)是概率编程的基础。`Distribution`类提供多样的概率统计接口,例如概率密度函数`pdf`、累积密度函数`cdf`、散度计算`kl_loss`、抽样`sample`等。现有的概率分布实例包括高斯分布,伯努利分布,指数型分布,几何分布和均匀分布。\n", "\n", "### 概率分布类\n", "\n", "- `Distribution`:所有概率分布的基类。\n", "\n", "- `Bernoulli`:伯努利分布。参数为试验成功的概率。\n", "\n", "- `Exponential`:指数型分布。参数为率参数。\n", "\n", "- `Geometric`:几何分布。参数为一次伯努利试验成功的概率。\n", "\n", "- `Normal`:正态(高斯)分布。参数为均值和标准差。\n", "\n", "- `Uniform`:均匀分布。参数为数轴上的最小值和最大值。\n", "\n", "- `Categorical`:类别分布。每种类别出现的概率。\n", "\n", "- `LogNormal`:对数正态分布。参数为位置参数和规模参数。\n", "\n", "- `Gumbel`: 耿贝尔极值分布。参数为位置参数和规模参数。\n", "\n", "- `Logistic`:逻辑斯谛分布。参数为位置参数和规模参数。\n", "\n", "- `Cauchy`:柯西分布。参数为位置参数和规模参数。\n", "\n", "#### Distribution基类\n", "\n", "`Distribution`是所有概率分布的基类。\n", "\n", "接口介绍:`Distribution`类支持的函数包括`prob`、`log_prob`、`cdf`、`log_cdf`、`survival_function`、`log_survival`、`mean`、`sd`、`var`、`entropy`、`kl_loss`、`cross_entropy`和`sample`。分布不同,所需传入的参数也不同。只有在派生类中才能使用,由派生类的函数实现决定参数。\n", "\n", "- `prob`:概率密度函数(PDF)/ 概率质量函数(PMF)。\n", "\n", "- `log_prob`:对数似然函数。\n", "\n", "- `cdf`:累积分布函数(CDF)。\n", "\n", "- `log_cdf`:对数累积分布函数。\n", "\n", "- `survival_function`:生存函数。\n", "\n", "- `log_survival`:对数生存函数。\n", "\n", "- `mean`:均值。\n", "\n", "- `sd`:标准差。\n", "\n", "- `var`:方差。\n", "\n", "- `entropy`:熵。\n", "\n", "- `kl_loss`:Kullback-Leibler 散度。\n", "\n", "- `cross_entropy`:两个概率分布的交叉熵。\n", "\n", "- `sample`:概率分布的随机抽样。\n", "\n", "- `get_dist_args`:概率分布在网络中使用的参数。\n", "\n", "- `get_dist_type`:概率分布的类型。\n", "\n", "#### 伯努利分布(Bernoulli)\n", "\n", "伯努利分布,继承自`Distribution`类。\n", "\n", "属性:\n", "\n", "- `Bernoulli.probs`:返回伯努利试验成功的概率,类型为`Tensor`。\n", "\n", "`Distribution`基类调用`Bernoulli`中私有接口以实现基类中的公有接口。`Bernoulli`支持的公有接口为:\n", "\n", "- `mean`,`mode`,`var`,`sd`:可选择传入试验成功的概率`probs1`。\n", "\n", "- `entropy`:可选择传入试验成功的概率`probs1`。\n", "\n", "- `cross_entropy`,`kl_loss`:必须传入`dist`和`probs1_b`。`dist`为另一分布的类型,目前只支持此处为“Bernoulli”。`probs1_b`为分布`b`的试验成功概率。可选择传入分布`a`的参数`probs1_a`。\n", "\n", "- `prob`,`log_prob`,`cdf`,`log_cdf`,`survival_function`,`log_survival`:必须传入`value`。可选择传入试验成功的概率`probs`。\n", "\n", "- `sample`:可选择传入样本形状`shape`和试验成功的概率`probs1`。\n", "\n", "- `get_dist_args`:可选择传入试验成功的概率`probs`。返回值为`(probs,)`,类型为tuple。\n", "\n", "- `get_dist_type`:返回“Bernoulli”。\n", "\n", "#### 指数分布(Exponential)\n", "\n", "指数分布,继承自`Distribution`类。\n", "\n", "属性:\n", "\n", "- `Exponential.rate`:返回分布的率参数,类型为`Tensor`。\n", "\n", "`Distribution`基类调用`Exponential`私有接口以实现基类中的公有接口。`Exponential`支持的公有接口为:\n", "\n", "- `mean`,`mode`,`var`,`sd`:可选择传入率参数`rate`。\n", "\n", "- `entropy`:可选择传入率参数`rate`。\n", "\n", "- `cross_entropy`,`kl_loss`:必须传入`dist`和`rate_b`。`dist`为另一分布的类型的名称, 目前只支持此处为“Exponential”。`rate_b`为分布`b`的率参数。可选择传入分布`a`的参数`rate_a`。\n", "\n", "- `prob`,`log_prob`,`cdf`,`log_cdf`,`survival_function`,`log_survival`:必须传入`value`。可选择传入率参数`rate`。\n", "\n", "- `sample`:可选择传入样本形状`shape`和率参数`rate`。返回值为`(rate,)`,类型为tuple。\n", "\n", "- `get_dist_args`:可选择传入率参数`rate`。返回值为`(rate,)`,类型为tuple。\n", "\n", "- `get_dist_type`:返回“Exponential”。\n", "\n", "#### 几何分布(Geometric)\n", "\n", "几何分布,继承自`Distribution`类。\n", "\n", "属性:\n", "\n", "- `Geometric.probs`:返回伯努利试验成功的概率,类型为`Tensor`。\n", "\n", "`Distribution`基类调用`Geometric`中私有接口以实现基类中的公有接口。`Geometric`支持的公有接口为:\n", "\n", "- `mean`,`mode`,`var`,`sd`:可选择传入试验成功的概率`probs1`。\n", "\n", "- `entropy`:可选择传入 试验成功的概率`probs1`。\n", "\n", "- `cross_entropy`,`kl_loss`:必须传入`dist`和`probs1_b`。`dist`为另一分布的类型的名称,目前只支持此处为“Geometric”。`probs1_b`为分布`b`的试验成功概率。可选择传入分布`a`的参数`probs1_a`。\n", "\n", "- `prob`,`log_prob`,`cdf`,`log_cdf`,`survival_function`,`log_survival`:必须传入`value`。可选择传入试验成功的概率`probs1`。\n", "\n", "- `sample`:可选择传入样本形状`shape`和试验成功的概率`probs1`。\n", "\n", "- `get_dist_args`:可选择传入试验成功的概率`probs1`。返回值为`(probs1,)`,类型为tuple。\n", "\n", "- `get_dist_type`:返回“Geometric”。\n", "\n", "#### 正态分布(Normal)\n", "\n", "正态(高斯)分布,继承自`Distribution`类。\n", "\n", "`Distribution`基类调用`Normal`中私有接口以实现基类中的公有接口。`Normal`支持的公有接口为:\n", "\n", "- `mean`,`mode`,`var`,`sd`:可选择传入分布的参数均值`mean`和标准差`sd`。\n", "\n", "- `entropy`:可选择传入分布的参数均值`mean`和标准差`sd`。\n", "\n", "- `cross_entropy`,`kl_loss`:必须传入`dist`,`mean_b`和`sd_b`。`dist`为另一分布的类型的名称,目前只支持此处为“Normal”。\n", "\n", "`mean_b`和`sd_b`为分布`b`的均值和标准差。可选择传入分布的参数`a`均值`mean_a`和标准差`sd_a`。\n", "\n", "- `prob`,`log_prob`,`cdf`,`log_cdf`,`survival_function`,`log_survival`:必须传入`value`。可选择分布的参数包括均值`mean_a`和标准差`sd_a`。\n", "\n", "- `sample`:可选择传入样本形状`shape`和分布的参数包括均值`mean_a`和标准差`sd_a`。\n", "\n", "- `get_dist_args`:可选择传入分布的参数均值`mean`和标准差`sd`。返回值为`(mean, sd)`,类型为tuple。\n", "\n", "- `get_dist_type`:返回“Normal”。\n", "\n", "#### 均匀分布(Uniform)\n", "\n", "均匀分布,继承自`Distribution`类。\n", "\n", "属性:\n", "\n", "- `Uniform.low`:返回分布的最小值,类型为`Tensor`。\n", "\n", "- `Uniform.high`:返回分布的最大值,类型为`Tensor`。\n", "\n", "`Distribution`基类调用`Uniform`以实现基类中的公有接口。`Uniform`支持的公有接口为:\n", "\n", "- `mean`,`mode`,`var`,`sd`:可选择传入分布的参数最大值`high`和最小值`low`。\n", "\n", "- `entropy`:可选择传入分布的参数最大值`high`和最小值`low`。\n", "\n", "- `cross_entropy`,`kl_loss`:必须传入`dist`,`high_b`和`low_b`。`dist`为另一分布的类型的名称,目前只支持此处为“Uniform”。`high_b`和`low_b`为分布`b`的参数。可选择传入分布`a`的参数即最大值`high_a`和最小值`low_a`。\n", "\n", "- `prob`,`log_prob`,`cdf`,`log_cdf`,`survival_function`,`log_survival`:必须传入`value`。可选择传入分布的参数最大值`high`和最小值`low`。\n", "\n", "- `sample`:可选择传入`shape`和分布的参数即最大值`high`和最小值`low`。\n", "\n", "- `get_dist_args`:可选择传入分布的参数最大值`high`和最小值`low`。返回值为`(low, high)`,类型为tuple。\n", "\n", "- `get_dist_type`:返回“Uniform”。\n", "\n", "#### 多类别分布(Categorical)\n", "\n", "多类别分布,继承自`Distribution`类。\n", "\n", "属性:\n", "\n", "- `Categorical.probs`:返回各种类别的概率,类型为`Tensor`。\n", "\n", "`Distribution`基类调用`Categorical`以实现基类中的公有接口。`Categorical`支持的公有接口为:\n", "\n", "- `mean`,`mode`,`var`,`sd`:可选择传入分布的参数类别概率`probs`。\n", "\n", "- `entropy`:可选择传入分布的参数类别概率`probs`。\n", "\n", "- `cross_entropy`,`kl_loss`:必须传入`dist`,`probs_b`。`dist`为另一分布的类型的名称,目前只支持此处为“Categorical”。`probs_b`为分布`b`的参数。可选择传入分布`a`的参数即`probs_a`。\n", "\n", "- `prob`,`log_prob`,`cdf`,`log_cdf`,`survival_function`,`log_survival`:必须传入`value`。可选择传入分布的参数类别概率`probs`。\n", "\n", "- `sample`:可选择传入`shape`和类别概率`probs`。\n", "\n", "- `get_dist_args`:可选择传入分布的参数类别概率`probs`。返回值为`(probs,)`,类型为tuple。\n", "\n", "- `get_dist_type`:返回“Categorical”。\n", "\n", "#### 对数正态分布(LogNormal)\n", "\n", "对数正态分布,继承自`TransformedDistribution`类,由`Exp`Bijector 和`Normal`Distribution 构成。\n", "\n", "属性:\n", "\n", "- `LogNormal.loc`:返回分布的位置参数,类型为`Tensor`。\n", "\n", "- `LogNormal.scale`:返回分布的规模参数,类型为`Tensor`。\n", "\n", "`Distribution`基类调用`LogNormal`及`TransformedDistribution`中私有接口以实现基类中的公有接口。`LogNormal`支持的公有接口为:\n", "\n", "- `mean`,`mode`,`var`,`sd`:可选择传入分布的位置参数`loc`和规模参数`scale`。\n", "\n", "- `entropy`:可选择传入分布的位置参数`loc`和规模参数`scale`。\n", "\n", "- `cross_entropy`,`kl_loss`:必须传入`dist`,`loc_b`和`scale_b`。`dist`为另一分布的类型的名称,目前只支持此处为“LogNormal”。`loc_b`和`scale_b`为分布`b`的均值和标准差。可选择传入分布的参数`a`均值`loc_a`和标准差`sclae_a`。\n", "\n", "- `prob`,`log_prob`,`cdf`,`log_cdf`,`survival_function`,`log_survival`:必须传入`value`。可选择分布的参数包括均值`loc_a`和标准差`scale_a`。`Distribution`基类调用`TransformedDistribution`私有接口。\n", "\n", "- `sample`:可选择传入样本形状`shape`和分布的参数包括均值`loc_a`和标准差`scale_a`。`Distribution`基类调用`TransformedDistribution`私有接口。\n", "\n", "- `get_dist_args`:可选择传入分布的位置参数`loc`和规模参数`scale`。返回值为`(loc, scale)`,类型为tuple。\n", "\n", "- `get_dist_type`:返回“LogNormal”。\n", "\n", "#### 柯西分布(Cauchy)\n", "\n", "柯西分布,继承自`Distribution`类。\n", "\n", "属性:\n", "\n", "- `Cauchy.loc`:返回分布的位置参数,类型为`Tensor`。\n", "\n", "- `Cauchy.scale`:返回分布的规模参数,类型为`Tensor`。\n", "\n", "`Distribution`基类调用`Cauchy`中私有接口以实现基类中的公有接口。`Cauchy`支持的公有接口为:\n", "\n", "- `entropy`:可选择传入分布的位置参数`loc`和规模参数`scale`。\n", "\n", "- `cross_entropy`,`kl_loss`:必须传入`dist`,`loc_b`和`scale_b`。`dist`为另一分布的类型的名称,目前只支持此处为“Cauchy”。`loc_b`和`scale_b`为分布`b`的位置参数和规模参数。可选择传入分布的参数`a`位置`loc_a`和规模`scale_a`。\n", "\n", "- `prob`,`log_prob`,`cdf`,`log_cdf`,`survival_function`,`log_survival`:必须传入`value`。可选择传入分布的位置参数`loc`和规模参数`scale`。\n", "\n", "- `sample`:可选择传入样本形状`shape`和分布的参数包括分布的位置参数`loc`和规模参数`scale`。\n", "\n", "- `get_dist_args`:可选择传入分布的位置参数`loc`和规模参数`scale`。返回值为`(loc, scale)`,类型为tuple。\n", "\n", "- `get_dist_type`:返回“Cauchy”。\n", "\n", "#### 耿贝尔极值分布(Gumbel)\n", "\n", "耿贝尔极值分布,继承自`TransformedDistribution`类,由`GumbelCDF`Bijector和`Uniform`Distribution 构成。\n", "\n", "属性:\n", "\n", "- `Gumbel.loc`:返回分布的位置参数,类型为`Tensor`。\n", "\n", "- `Gumbel.scale`:返回分布的规模参数,类型为`Tensor`。\n", "\n", "`Distribution`基类调用`Gumbel`中私有接口以实现基类中的公有接口。`Gumbel`支持的公有接口为:\n", "\n", "- `mean`,`mode`,`var`,`sd`:无参数。\n", "\n", "- `entropy`:无参数。\n", "\n", "- `cross_entropy`,`kl_loss`:必须传入`dist`,`loc_b`和`scale_b`。`dist`为另一分布的类型的名称,目前只支持此处为“Gumbel”。`loc_b`和`scale_b`为分布`b`的位置参数和规模参数。\n", "\n", "- `prob`,`log_prob`,`cdf`,`log_cdf`,`survival_function`,`log_survival`:必须传入`value`。\n", "\n", "- `sample`:可选择传入样本形状`shape`。\n", "\n", "- `get_dist_args`:可选择传入分布的位置参数`loc`和规模参数`scale`。返回值为`(loc, scale)`,类型为tuple。\n", "\n", "- `get_dist_type`:返回“Gumbel”。\n", "\n", "#### 逻辑斯谛分布(Logistic)\n", "\n", "逻辑斯谛分布,继承自`Distribution`类。\n", "\n", "属性:\n", "\n", "- `Logistic.loc`:返回分布的位置参数,类型为`Tensor`。\n", "\n", "- `Logistic.scale`:返回分布的规模参数,类型为`Tensor`。\n", "\n", "`Distribution`基类调用`logistic`中私有接口以实现基类中的公有接口。`Logistic`支持的公有接口为:\n", "\n", "- `mean`,`mode`,`var`,`sd`:可选择传入分布的位置参数`loc`和规模参数`scale`。\n", "\n", "- `entropy`:可选择传入分布的位置参数`loc`和规模参数`scale`。\n", "\n", "- `prob`,`log_prob`,`cdf`,`log_cdf`,`survival_function`,`log_survival`:必须传入`value`。可选择传入分布的位置参数`loc`和规模参数`scale`。\n", "\n", "- `sample`:可选择传入样本形状`shape`和分布的参数包括分布的位置参数`loc`和规模参数`scale`。\n", "\n", "- `get_dist_args`:可选择传入分布的位置参数`loc`和规模参数`scale`。返回值为`(loc, scale)`,类型为tuple。\n", "\n", "- `get_dist_type`:返回“Logistic”。\n", "\n", "#### 泊松分布\n", "\n", "泊松分布,继承自`Distribution`类。\n", "\n", "属性:\n", "\n", "- `Poisson.rate`:返回分布的率参数,类型为Tensor。\n", "\n", "`Distribution` 基类调用`Poisson`中私有接口以实现基类中的公有接口。`Poisson`支持的公有接口为:\n", "\n", "- `mean`,`mode`,`var`,`sd`:可选择传入分布的率参数 rate 。\n", "\n", "- `prob`,`log_prob`,`cdf`,`log_cdf`,`survival_function`,`log_survival`:必须传入`value`。可选择传入分布的率参数`rate`。\n", "\n", "- `sample`:可选择传入样本形状shape 和分布的率参数 rate 。\n", "\n", "- `get_dist_args`:可选择传入分布的率参数`rate`。返回值为`(rate,)`,类型为tuple。\n", "\n", "- `get_dist_type`:返回“Poisson”。\n", "\n", "#### 伽马分布(Gamma)\n", "\n", "伽马分布,继承自 `Distribution` 类。\n", "\n", "属性:\n", "\n", "- `Gamma.concentration`:返回分布的参数 `concentration` ,类型为`Tensor`。\n", "\n", "- `Gamma.rate`:返回分布的参数 `rate` ,类型为`Tensor`。\n", "\n", "`Distribution` 基类调用 `Gamma` 中私有接口以实现基类中的公有接口。`Gamma` 支持的公有接口为:\n", "\n", "- `mean`,`mode`,`sd`,`var`:可选择传入分布的参数`concentration`和参数`rate` 。\n", "\n", "- `entropy`:可选择传入分布的参数`concentration`和参数`rate`。\n", "\n", "- `prob`,`log_prob`,`cdf`,`log_cdf`,`survival_function`,`log_survival`:必须传入`value`。可选择传入分布的参数`concentration`和参数`rate`。\n", "\n", "- `cross_entropy`,`kl_loss`:必须传入`dist`,`concentration_b`和`rate_b`。`dist`为另一分布的类型的名称,目前只支持此处为“Gamma”。 `concentration_b`和`rate_b`为分布`b`的参数。可选择传入分布`a`的参数即`concentration_a`和`rate_a`。\n", "\n", "- `sample`:可选择传入样本形状`shape`和分布的参数包括分布的参数`concentration`和参数`rate`。\n", "\n", "- `get_dist_args`:可选择传入分布的参数`concentration`和参数`rate`。返回值为`(concentration, rate)`,类型为tuple。\n", "\n", "- `get_dist_type`:返回“Gamma”。\n", "\n", "#### 贝塔分布(Beta)\n", "\n", "贝塔分布,继承自 `Distribution` 类。\n", "\n", "属性:\n", "\n", "- `Beta.concentration1`:返回分布的参数 `concentration1` ,类型为`Tensor`。\n", "\n", "- `Beta.concentration0`:返回分布的参数 `concentration0` ,类型为`Tensor`。\n", "\n", "`Distribution` 基类调用 `Beta` 中私有接口以实现基类中的公有接口。`Beta` 支持的公有接口为:\n", "\n", "- `mean`,`mode`,`sd`,`var`:可选择传入分布的参数`concentration1`和参数`concentration0`。\n", "\n", "- `entropy`:可选择传入分布的参数`concentration1`和参数`concentration0`。\n", "\n", "- `prob`,`log_prob`:必须传入`value`。可选择传入分布的参数`concentration1`和参数`concentration0`。\n", "\n", "- `cross_entropy`,`kl_loss`:必须传入`dist`,`concentration1_b`和`concentration1_b`。`dist`为另一分布的类型的名称,目前只支持此处为“Beta”。`concentration1_b`和`concentration1_b`为分布`b`的参数。可选择传入分布`a`的参数即`concentration1_a`和`concentration0_a`。\n", "\n", "- `sample`:可选择传入样本形状`shape`和分布的参数包括分布的位置参数`loc`和规模参数`scale`。\n", "\n", "- `get_dist_args`:可选择传入分布的参数`concentration1`和参数`concentration0`。返回值为`(concentration1, concentration0)`,类型为tuple。\n", "\n", "- `get_dist_type`:返回“Beta”。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 概率分布类在PyNative模式下的应用\n", "\n", "`Distribution`子类可在PyNative模式下使用。\n", "\n", "以`Normal`为例,创建一个均值为0.0、标准差为1.0的正态分布,然后计算相关函数。" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "mean: 0.0\n", "var: 1.0\n", "entropy: 1.4189385\n", "prob: [0.35206532 0.3989423 0.35206532]\n", "cdf: [0.30853754 0.5 0.69146246]\n", "kl: 0.44314718\n", "dist_arg: (Tensor(shape=[], dtype=Float32, value= 0), Tensor(shape=[], dtype=Float32, value= 1))\n" ] } ], "source": [ "import mindspore as ms\n", "import mindspore.nn.probability.distribution as msd\n", "\n", "ms.set_context(mode=ms.PYNATIVE_MODE, device_target=\"GPU\")\n", "\n", "my_normal = msd.Normal(0.0, 1.0, dtype=ms.float32)\n", "\n", "mean = my_normal.mean()\n", "var = my_normal.var()\n", "entropy = my_normal.entropy()\n", "\n", "value = ms.Tensor([-0.5, 0.0, 0.5], dtype=ms.float32)\n", "prob = my_normal.prob(value)\n", "cdf = my_normal.cdf(value)\n", "\n", "mean_b = ms.Tensor(1.0, dtype=ms.float32)\n", "sd_b = ms.Tensor(2.0, dtype=ms.float32)\n", "kl = my_normal.kl_loss('Normal', mean_b, sd_b)\n", "\n", "# get the distribution args as a tuple\n", "dist_arg = my_normal.get_dist_args()\n", "\n", "print(\"mean: \", mean)\n", "print(\"var: \", var)\n", "print(\"entropy: \", entropy)\n", "print(\"prob: \", prob)\n", "print(\"cdf: \", cdf)\n", "print(\"kl: \", kl)\n", "print(\"dist_arg: \", dist_arg)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 概率分布类在图模式下的应用\n", "\n", "在图模式下,`Distribution`子类可用在网络中。" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "pdf: [0.35206532 0.3989423 0.35206532]\n", "kl: 0.5\n" ] } ], "source": [ "import mindspore.nn as nn\n", "import mindspore as ms\n", "import mindspore.nn.probability.distribution as msd\n", "ms.set_context(mode=ms.GRAPH_MODE)\n", "\n", "class Net(nn.Cell):\n", " def __init__(self):\n", " super(Net, self).__init__()\n", " self.normal = msd.Normal(0.0, 1.0, dtype=ms.float32)\n", "\n", " def construct(self, value, mean, sd):\n", " pdf = self.normal.prob(value)\n", " kl = self.normal.kl_loss(\"Normal\", mean, sd)\n", " return pdf, kl\n", "\n", "net = Net()\n", "value = ms.Tensor([-0.5, 0.0, 0.5], dtype=ms.float32)\n", "mean = ms.Tensor(1.0, dtype=ms.float32)\n", "sd = ms.Tensor(1.0, dtype=ms.float32)\n", "pdf, kl = net(value, mean, sd)\n", "print(\"pdf: \", pdf)\n", "print(\"kl: \", kl)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### TransformedDistribution类接口设计\n", "\n", "`TransformedDistribution`继承自`Distribution`,是可通过映射f(x)变化得到的数学分布的基类。其接口包括:\n", "\n", "1. 属性\n", "\n", " - `bijector`:返回分布的变换方法。\n", "\n", " - `distribution`:返回原始分布。\n", "\n", " - `is_linear_transformation`:返回线性变换标志。\n", "\n", "2. 接口函数(以下接口函数的参数与构造函数中`distribution`的对应接口的参数相同)。\n", "\n", " - `cdf`:累积分布函数(CDF)。\n", "\n", " - `log_cdf`:对数累积分布函数。\n", "\n", " - `survival_function`:生存函数。\n", "\n", " - `log_survival`:对数生存函数。\n", "\n", " - `prob`:概率密度函数(PDF)/ 概率质量函数(PMF)。\n", "\n", " - `log_prob`:对数似然函数。\n", "\n", " - `sample`:随机取样。\n", "\n", " - `mean`:无参数。只有当`Bijector.is_constant_jacobian=true`时可调用。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### PyNative模式下调用TransformedDistribution实例\n", "\n", "`TransformedDistribution`子类可在PyNative模式下使用。\n", "\n", "这里构造一个`TransformedDistribution`实例,使用`Normal`分布作为需要变换的分布类,使用`Exp`作为映射变换,可以生成`LogNormal`分布。" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "TransformedDistribution<\n", " (_bijector): Exp\n", " (_distribution): Normal\n", " >\n", "underlying distribution:\n", " Normal\n", "bijector:\n", " Exp\n", "cdf:\n", " [0.7558914 0.9462397 0.9893489]\n", "sample:\n", " (3, 2)\n" ] } ], "source": [ "import numpy as np\n", "import mindspore.nn as nn\n", "import mindspore.nn.probability.bijector as msb\n", "import mindspore.nn.probability.distribution as msd\n", "import mindspore as ms\n", "\n", "ms.set_context(mode=ms.PYNATIVE_MODE)\n", "\n", "normal = msd.Normal(0.0, 1.0, dtype=ms.float32)\n", "exp = msb.Exp()\n", "LogNormal = msd.TransformedDistribution(exp, normal, seed=0, name=\"LogNormal\")\n", "\n", "# compute cumulative distribution function\n", "x = np.array([2.0, 5.0, 10.0], dtype=np.float32)\n", "tx = ms.Tensor(x, dtype=ms.float32)\n", "cdf = LogNormal.cdf(tx)\n", "\n", "# generate samples from the distribution\n", "shape = ((3, 2))\n", "sample = LogNormal.sample(shape)\n", "\n", "# get information of the distribution\n", "print(LogNormal)\n", "# get information of the underlying distribution and the bijector separately\n", "print(\"underlying distribution:\\n\", LogNormal.distribution)\n", "print(\"bijector:\\n\", LogNormal.bijector)\n", "# get the computation results\n", "print(\"cdf:\\n\", cdf)\n", "print(\"sample:\\n\", sample.shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "当构造`TransformedDistribution`映射变换的`is_constant_jacobian = true`时(如`ScalarAffine`),构造的`TransformedDistribution`实例可以使用直接使用`mean`接口计算均值,例如:" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2.0\n" ] } ], "source": [ "normal = msd.Normal(0.0, 1.0, dtype=ms.float32)\n", "scalaraffine = msb.ScalarAffine(1.0, 2.0)\n", "trans_dist = msd.TransformedDistribution(scalaraffine, normal, seed=0)\n", "mean = trans_dist.mean()\n", "print(mean)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 图模式下调用TransformedDistribution实例\n", "\n", "在图模式下,`TransformedDistribution`类可用在网络中。" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "cdf: [0.7558914 0.86403143 0.9171715 0.9462397 ]\n", "sample: (2, 3)\n" ] } ], "source": [ "import numpy as np\n", "import mindspore.nn as nn\n", "import mindspore as ms\n", "import mindspore.nn.probability.bijector as msb\n", "import mindspore.nn.probability.distribution as msd\n", "ms.set_context(mode=ms.GRAPH_MODE)\n", "\n", "class Net(nn.Cell):\n", " def __init__(self, shape, dtype=ms.float32, seed=0, name='transformed_distribution'):\n", " super(Net, self).__init__()\n", " # create TransformedDistribution distribution\n", " self.exp = msb.Exp()\n", " self.normal = msd.Normal(0.0, 1.0, dtype=dtype)\n", " self.lognormal = msd.TransformedDistribution(self.exp, self.normal, seed=seed, name=name)\n", " self.shape = shape\n", "\n", " def construct(self, value):\n", " cdf = self.lognormal.cdf(value)\n", " sample = self.lognormal.sample(self.shape)\n", " return cdf, sample\n", "\n", "shape = (2, 3)\n", "net = Net(shape=shape, name=\"LogNormal\")\n", "x = np.array([2.0, 3.0, 4.0, 5.0]).astype(np.float32)\n", "tx = ms.Tensor(x, dtype=ms.float32)\n", "cdf, sample = net(tx)\n", "print(\"cdf: \", cdf)\n", "print(\"sample: \", sample.shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 概率分布映射\n", "\n", "Bijector(`mindspore.nn.probability.bijector`)是概率编程的基本组成部分。Bijector描述了一种随机变量的变换方法,可以通过一个已有的随机变量X和一个映射函数f生成一个新的随机变量$Y = f(x)$。\n", "\n", "`Bijector`提供了映射相关的四种变换方法。它可以当做算子直接使用,也可以作用在某个随机变量`Distribution`类实例上生成新的随机变量的`Distribution`类实例。\n", "\n", "### Bijector类接口设计\n", "\n", "#### Bijector基类\n", "\n", "`Bijector`类是所有概率分布映射的基类。其接口包括:\n", "\n", "1. 属性\n", "\n", " - `name`:返回`name`的值。\n", "\n", " - `is_dtype`:返回`dtype`的值。\n", "\n", " - `parameter`:返回`parameter`的值。\n", "\n", " - `is_constant_jacobian`:返回`is_constant_jacobian`的值。\n", "\n", " - `is_injective`:返回`is_injective`的值。\n", "\n", "2. 映射函数\n", "\n", " - `forward`:正向映射,创建派生类后由派生类的`_forward`决定参数。\n", "\n", " - `inverse`:反向映射,创建派生类后由派生类的`_inverse`决定参数。\n", "\n", " - `forward_log_jacobian`:正向映射的导数的对数,创建派生类后由派生类的`_forward_log_jacobian`决定参数。\n", "\n", " - `inverse_log_jacobian`:反向映射的导数的对数,创建派生类后由派生类的`_inverse_log_jacobian`决定参数。\n", "\n", "`Bijector`作为函数调用:输入是一个`Distribution`类:生成一个`TransformedDistribution` *(不可在图内调用)*。\n", "\n", "#### 幂函数变换映射(PowerTransform)\n", "\n", "`PowerTransform`做如下变量替换:$Y = g(X) = {(1 + X \\times power)}^{1 / power}$。其接口包括:\n", "\n", "1. 属性\n", "\n", " - `power`:返回`power`的值,类型为`Tensor`。\n", "\n", "2. 映射函数\n", "\n", " - `forward`:正向映射,输入为`Tensor`。\n", "\n", " - `inverse`:反向映射,输入为`Tensor`。\n", "\n", " - `forward_log_jacobian`:正向映射的导数的对数,输入为`Tensor`。\n", "\n", " - `inverse_log_jacobian`:反向映射的导数的对数,输入为`Tensor`。\n", "\n", "#### 指数变换映射(Exp)\n", "\n", "`Exp`做如下变量替换:$Y = g(X)= exp(X)$。其接口包括:\n", "\n", "映射函数\n", "\n", "- `forward`:正向映射,输入为`Tensor`。\n", "\n", "- `inverse`:反向映射,输入为`Tensor`。\n", "\n", "- `forward_log_jacobian`:正向映射的导数的对数,输入为`Tensor`。\n", "\n", "- `inverse_log_jacobian`:反向映射的导数的对数,输入为`Tensor`。\n", "\n", "#### 标量仿射变换映射(ScalarAffine)\n", "\n", "`ScalarAffine`做如下变量替换:$Y = g(X) = scale\\times X + shift$。其接口包括:\n", "\n", "1. 属性\n", "\n", " - `scale`:返回`scale`的值,类型为`Tensor`。\n", "\n", " - `shift`:返回`shift`的值,类型为`Tensor`。\n", "\n", "2. 映射函数\n", "\n", " - `forward`:正向映射,输入为`Tensor`。\n", "\n", " - `inverse`:反向映射,输入为`Tensor`。\n", "\n", " - `forward_log_jacobian`:正向映射的导数的对数,输入为`Tensor`。\n", "\n", " - `inverse_log_jacobian`:反向映射的导数的对数,输入为`Tensor`。\n", "\n", "#### Softplus变换映射(Softplus)\n", "\n", "`Softplus`做如下变量替换:$Y = g(X) = \\frac{log(1 + e ^ {sharpness \\times X}\\ \\ \\ \\ \\ \\ )} {sharpness}$。其接口包括:\n", "\n", "1. 属性\n", "\n", " - `sharpness`:返回`sharpness`的值,类型为`Tensor`。\n", "\n", "2. 映射函数\n", "\n", " - `forward`:正向映射,输入为`Tensor`。\n", "\n", " - `inverse`:反向映射,输入为`Tensor`。\n", "\n", " - `forward_log_jacobian`:正向映射的导数的对数,输入为`Tensor`。\n", "\n", " - `inverse_log_jacobian`:反向映射的导数的对数,输入为`Tensor`。\n", "\n", "#### 耿贝尔累计密度函数映射(GumbelCDF)\n", "\n", "`GumbelCDF`做如下变量替换:$Y = g(X) = \\exp(-\\exp(-\\frac{X - loc}{scale}))$。其接口包括:\n", "\n", "1. 属性\n", "\n", " - `loc`:返回`loc`的值,类型为`Tensor`。\n", "\n", " - `scale`:返回`scale`的值,类型为`Tensor`。\n", "\n", "2. 映射函数\n", "\n", " - `forward`:正向映射,输入为`Tensor`。\n", "\n", " - `inverse`:反向映射,输入为`Tensor`。\n", "\n", " - `forward_log_jacobian`:正向映射的导数的对数,输入为`Tensor`。\n", "\n", " - `inverse_log_jacobian`:反向映射的导数的对数,输入为`Tensor`。\n", "\n", "#### 逆映射(Invert)\n", "\n", "`Invert`对一个映射做逆变换,其接口包括:\n", "\n", "1. 属性\n", "\n", " - `bijector`:返回初始化时使用的`Bijector`,类型为`Bijector`。\n", "\n", "2. 映射函数\n", "\n", " - `forward`:正向映射,输入为`Tensor`。\n", "\n", " - `inverse`:反向映射,输入为`Tensor`。\n", "\n", " - `forward_log_jacobian`:正向映射的导数的对数,输入为`Tensor`。\n", "\n", " - `inverse_log_jacobian`:反向映射的导数的对数,输入为`Tensor`。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### PyNative模式下调用Bijector实例\n", "\n", "在执行之前,我们需要导入需要的库文件包。双射类最主要的库是`mindspore.nn.probability.bijector`,导入后我们使用`msb`作为库的缩写并进行调用。\n", "\n", "下面我们以`PowerTransform`为例。创建一个指数为2的`PowerTransform`对象。" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "PowerTransform\n", "forward: [2.236068 2.6457515 3. 3.3166249]\n", "inverse: [ 1.5 4. 7.5 12.000001]\n", "forward_log_jacobian: [-0.804719 -0.9729551 -1.0986123 -1.1989477]\n", "inverse_log_jacobian: [0.6931472 1.0986123 1.3862944 1.609438 ]\n" ] } ], "source": [ "import numpy as np\n", "import mindspore.nn as nn\n", "import mindspore.nn.probability.bijector as msb\n", "import mindspore as ms\n", "\n", "ms.set_context(mode=ms.PYNATIVE_MODE)\n", "\n", "powertransform = msb.PowerTransform(power=2.)\n", "\n", "x = np.array([2.0, 3.0, 4.0, 5.0], dtype=np.float32)\n", "tx = ms.Tensor(x, dtype=ms.float32)\n", "forward = powertransform.forward(tx)\n", "inverse = powertransform.inverse(tx)\n", "forward_log_jaco = powertransform.forward_log_jacobian(tx)\n", "inverse_log_jaco = powertransform.inverse_log_jacobian(tx)\n", "\n", "print(powertransform)\n", "print(\"forward: \", forward)\n", "print(\"inverse: \", inverse)\n", "print(\"forward_log_jacobian: \", forward_log_jaco)\n", "print(\"inverse_log_jacobian: \", inverse_log_jaco)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 图模式下调用Bijector实例\n", "\n", "在图模式下,`Bijector`子类可用在网络中。" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "forward: [2.236068 2.6457515 3. 3.3166249]\n", "inverse: [ 1.5 4. 7.5 12.000001]\n", "forward_log_jacobian: [-0.804719 -0.9729551 -1.0986123 -1.1989477]\n", "inverse_log_jacobian: [0.6931472 1.0986123 1.3862944 1.609438 ]\n" ] } ], "source": [ "import numpy as np\n", "import mindspore.nn as nn\n", "import mindspore as ms\n", "import mindspore.nn.probability.bijector as msb\n", "ms.set_context(mode=ms.GRAPH_MODE)\n", "\n", "class Net(nn.Cell):\n", " def __init__(self):\n", " super(Net, self).__init__()\n", " # create a PowerTransform bijector\n", " self.powertransform = msb.PowerTransform(power=2.)\n", "\n", " def construct(self, value):\n", " forward = self.powertransform.forward(value)\n", " inverse = self.powertransform.inverse(value)\n", " forward_log_jaco = self.powertransform.forward_log_jacobian(value)\n", " inverse_log_jaco = self.powertransform.inverse_log_jacobian(value)\n", " return forward, inverse, forward_log_jaco, inverse_log_jaco\n", "\n", "net = Net()\n", "x = np.array([2.0, 3.0, 4.0, 5.0]).astype(np.float32)\n", "tx = ms.Tensor(x, dtype=ms.float32)\n", "forward, inverse, forward_log_jaco, inverse_log_jaco = net(tx)\n", "print(\"forward: \", forward)\n", "print(\"inverse: \", inverse)\n", "print(\"forward_log_jacobian: \", forward_log_jaco)\n", "print(\"inverse_log_jacobian: \", inverse_log_jaco)" ] } ], "metadata": { "kernelspec": { "display_name": "MindSpore", "language": "python", "name": "mindspore" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.6" } }, "nbformat": 4, "nbformat_minor": 4 }