Jekyll2022-10-14T02:14:27+00:00https://yangfangs.github.io//feed.xml小小羊小小羊|学会记录和分享Yang Fang科研彩图常用的配色方案-ColorBrewer的使用2020-03-13T00:00:00+00:002020-03-13T00:00:00+00:00https://yangfangs.github.io//2020/03/13/science-paper-colors<p>前段时间师兄推荐一个科研图常用配色系统<a href="https://colorbrewer2.org/">Colorbrewer</a>非常不错,好的彩图配色方案能增加你文章的整体的观感,更容易让人接受能。好的彩色配色方案应该具有高的区分度,
能让人一眼能分清 legend 展示和相对应图中各个曲线或者柱状图。后来发现到 ggplot2 和 Adobe illustrator 中都有相应的调色版非常方便,记录下使用方法。</p>
<h1 id="1-使用colorbrewer网站获取颜色">1 使用Colorbrewer网站获取颜色</h1>
<h2 id="colorbrewer20网站">Colorbrewer2.0网站</h2>
<p><img src="/images/posts/R/ColorBrewer_Color.png" alt="Colorbrewer" /></p>
<h2 id="colorbrewer20使用说明">Colorbrewer2.0使用说明</h2>
<ul>
<li>sequential: 连续型单渐变色,色彩由一种颜色并且从浅到深排列。这种色彩组合适用于连续型数据展示。</li>
<li>diverging: 离散型双渐变色,色彩由一种到另一种颜色的高区度排列。这种色彩组合适用于离散数据的展示。</li>
<li>qualitative: 高区分度色。色彩由区分度极高的颜色组成。这种颜色适用于高区分度曲线或者柱状图使用。</li>
<li>Colorbrewer2.0 最多提供12种颜色组合,基本能应付所有画图组合。</li>
<li>Colorbrewer2.0 提供的十六进制的色彩代码例如 <code class="language-plaintext highlighter-rouge">#762a83</code> 和 <code class="language-plaintext highlighter-rouge">#5aae61</code></li>
</ul>
<h1 id="2-使用-r-语言的-rcolorbrewer包提供的色彩方案">2 使用 R 语言的 <a href="https://cran.r-project.org/web/packages/RColorBrewer/index.html">RColorBrewer</a>包提供的色彩方案</h1>
<h2 id="使用-displaybrewerall-查看-rcolorbrewer-包的调色版">使用 display.brewer.all() 查看 RColorBrewer 包的调色版</h2>
<div class="language-R highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">></span><span class="w"> </span><span class="n">library</span><span class="p">(</span><span class="n">RColorBrewer</span><span class="p">)</span><span class="w">
</span><span class="o">></span><span class="w"> </span><span class="n">display.brewer.all</span><span class="p">()</span><span class="w">
</span></code></pre></div></div>
<p><img src="/images/posts/R/RColorBrewer_palettes.png" alt="Colorbrewer" /></p>
<h2 id="使用-brewerpalinfo-命令查看色彩系统">使用 brewer.pal.info 命令查看色彩系统。</h2>
<div class="language-R highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">></span><span class="w"> </span><span class="n">brewer.pal.info</span><span class="w">
</span><span class="n">maxcolors</span><span class="w"> </span><span class="n">category</span><span class="w"> </span><span class="n">colorblind</span><span class="w">
</span><span class="n">BrBG</span><span class="w"> </span><span class="m">11</span><span class="w"> </span><span class="n">div</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">PiYG</span><span class="w"> </span><span class="m">11</span><span class="w"> </span><span class="n">div</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">PRGn</span><span class="w"> </span><span class="m">11</span><span class="w"> </span><span class="n">div</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">PuOr</span><span class="w"> </span><span class="m">11</span><span class="w"> </span><span class="n">div</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">RdBu</span><span class="w"> </span><span class="m">11</span><span class="w"> </span><span class="n">div</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">RdGy</span><span class="w"> </span><span class="m">11</span><span class="w"> </span><span class="n">div</span><span class="w"> </span><span class="kc">FALSE</span><span class="w">
</span><span class="n">RdYlBu</span><span class="w"> </span><span class="m">11</span><span class="w"> </span><span class="n">div</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">RdYlGn</span><span class="w"> </span><span class="m">11</span><span class="w"> </span><span class="n">div</span><span class="w"> </span><span class="kc">FALSE</span><span class="w">
</span><span class="n">Spectral</span><span class="w"> </span><span class="m">11</span><span class="w"> </span><span class="n">div</span><span class="w"> </span><span class="kc">FALSE</span><span class="w">
</span><span class="n">Accent</span><span class="w"> </span><span class="m">8</span><span class="w"> </span><span class="n">qual</span><span class="w"> </span><span class="kc">FALSE</span><span class="w">
</span><span class="n">Dark2</span><span class="w"> </span><span class="m">8</span><span class="w"> </span><span class="n">qual</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">Paired</span><span class="w"> </span><span class="m">12</span><span class="w"> </span><span class="n">qual</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">Pastel1</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">qual</span><span class="w"> </span><span class="kc">FALSE</span><span class="w">
</span><span class="n">Pastel2</span><span class="w"> </span><span class="m">8</span><span class="w"> </span><span class="n">qual</span><span class="w"> </span><span class="kc">FALSE</span><span class="w">
</span><span class="n">Set1</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">qual</span><span class="w"> </span><span class="kc">FALSE</span><span class="w">
</span><span class="n">Set2</span><span class="w"> </span><span class="m">8</span><span class="w"> </span><span class="n">qual</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">Set3</span><span class="w"> </span><span class="m">12</span><span class="w"> </span><span class="n">qual</span><span class="w"> </span><span class="kc">FALSE</span><span class="w">
</span><span class="n">Blues</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">seq</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">BuGn</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">seq</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">BuPu</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">seq</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">GnBu</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">seq</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">Greens</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">seq</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">Greys</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">seq</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">Oranges</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">seq</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">OrRd</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">seq</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">PuBu</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">seq</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">PuBuGn</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">seq</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">PuRd</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">seq</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">Purples</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">seq</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">RdPu</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">seq</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">Reds</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">seq</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">YlGn</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">seq</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">YlGnBu</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">seq</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">YlOrBr</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">seq</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span><span class="n">YlOrRd</span><span class="w"> </span><span class="m">9</span><span class="w"> </span><span class="n">seq</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
</span></code></pre></div></div>
<ul>
<li>从上到下依次是 diverging, qualitative, sequential 分类以及每类颜色个个数。</li>
</ul>
<h2 id="使用-displaybrewerpal-查看具体每个配色方案">使用 display.brewer.pal() 查看具体每个配色方案</h2>
<div class="language-R highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">display.brewer.pal</span><span class="p">(</span><span class="m">8</span><span class="p">,</span><span class="s2">"Set1"</span><span class="p">)</span><span class="w">
</span></code></pre></div></div>
<p><img src="/images/posts/R/colorbrewer_package_each_color.png" alt="colorbrewer_package" /></p>
<h2 id="使用-brewerpal8set1-获取十六进制颜色代码">使用 brewer.pal(8,”Set1”) 获取十六进制颜色代码</h2>
<div class="language-R highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="w"> </span><span class="o">></span><span class="w"> </span><span class="n">brewer.pal</span><span class="p">(</span><span class="m">8</span><span class="p">,</span><span class="s2">"Set1"</span><span class="p">)</span><span class="w">
</span><span class="p">[</span><span class="m">1</span><span class="p">]</span><span class="w"> </span><span class="s2">"#E41A1C"</span><span class="w"> </span><span class="s2">"#377EB8"</span><span class="w"> </span><span class="s2">"#4DAF4A"</span><span class="w"> </span><span class="s2">"#984EA3"</span><span class="w"> </span><span class="s2">"#FF7F00"</span><span class="w"> </span><span class="s2">"#FFFF33"</span><span class="w"> </span><span class="s2">"#A65628"</span><span class="w"> </span><span class="s2">"#F781BF"</span><span class="w">
</span></code></pre></div></div>
<h2 id="使用-colorramppalette-自定义颜色数量">使用 colorRampPalette() 自定义颜色数量</h2>
<div class="language-R highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">cols</span><span class="o"><-</span><span class="n">brewer.pal</span><span class="p">(</span><span class="m">8</span><span class="p">,</span><span class="w"> </span><span class="s2">"YlGnBu"</span><span class="p">)</span><span class="w">
</span><span class="n">pal</span><span class="o"><-</span><span class="n">colorRampPalette</span><span class="p">(</span><span class="n">cols</span><span class="p">)</span><span class="w">
</span><span class="n">mycolors</span><span class="o"><-</span><span class="n">pal</span><span class="p">(</span><span class="m">20</span><span class="p">)</span><span class="w">
</span><span class="n">image</span><span class="p">(</span><span class="m">1</span><span class="w"> </span><span class="o">:</span><span class="w"> </span><span class="nf">length</span><span class="p">(</span><span class="n">mycolors</span><span class="p">),</span><span class="w"> </span><span class="m">1</span><span class="p">,</span><span class="w"> </span><span class="n">as.matrix</span><span class="p">(</span><span class="m">1</span><span class="w"> </span><span class="o">:</span><span class="w"> </span><span class="nf">length</span><span class="p">(</span><span class="n">mycolors</span><span class="p">)),</span><span class="w"> </span><span class="n">col</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">mycolors</span><span class="w"> </span><span class="p">,</span><span class="n">xlab</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">""</span><span class="p">,</span><span class="w"> </span><span class="n">ylab</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">""</span><span class="p">,</span><span class="w"> </span><span class="n">xaxt</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"n"</span><span class="p">,</span><span class="w"> </span><span class="n">yaxt</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"n"</span><span class="p">,</span><span class="w"> </span><span class="n">bty</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"n"</span><span class="p">)</span><span class="w">
</span></code></pre></div></div>
<p><img src="/images/posts/R/colorRampPalette_self.png" alt="colorRampPalette_self.png" /></p>
<h1 id="3-在adobe-illustrator-中使用-colorbrewer-调色版">3 在Adobe Illustrator 中使用 ColorBrewer 调色版</h1>
<ul>
<li>AI中直接配有科学色彩库方便科学用色,依次菜单窗口—>色板—>色板库—>科学,选择需要的颜色。</li>
</ul>
<p><img src="/images/posts/R/AI_color.png" alt="AI_color.png" /></p>
<h1 id="参考">参考</h1>
<ul>
<li><a href="https://bookdown.org/rdpeng/exdata/plotting-and-color-in-r.html">Exploratory Data Analysis with R</a></li>
<li><a href="https://www.cnblogs.com/shaocf/p/9600340.html">使用 ggplot2 和 RColorBrewer 扩展调色板</a></li>
<li><a href="https://www.jianshu.com/p/a8856757a0d2">RColorBrewer与ggplot2</a></li>
</ul>Yang Fang前段时间师兄推荐一个科研图常用配色系统Colorbrewer非常不错,好的彩图配色方案能增加你文章的整体的观感,更容易让人接受能。好的彩色配色方案应该具有高的区分度, 能让人一眼能分清 legend 展示和相对应图中各个曲线或者柱状图。后来发现到 ggplot2 和 Adobe illustrator 中都有相应的调色版非常方便,记录下使用方法。计算1亿对Hamming distance仅需5ms的Java源码bitCount解析2019-04-28T00:00:00+00:002019-04-28T00:00:00+00:00https://yangfangs.github.io//2019/04/28/java-bitcount-test<p>汉明距离(<a href="https://en.wikipedia.org/wiki/Hamming_distance">Hamming Distance</a>)的基本思想很简单,就是找不同。当求由01二进制组成
的向量间的汉明距离可以由位运算直接进行,速度非常快。好的算法能让计算速度达到极快,Java内置bitCount源码就实现了一种速度极快的算法。在Linux机器
(CPU: i7-4790 @ 3.6GHz)测试了1亿对汉明距离只用0.5ms,是普通算法的196倍。</p>
<h1 id="测试1亿对汉明距离结果">测试1亿对汉明距离结果</h1>
<p>测试基于32位的01二进制数进行的。</p>
<p>测试类:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="kn">import</span> <span class="nn">java.util.Arrays</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.Random</span><span class="o">;</span>
<span class="kd">public</span> <span class="kd">class</span> <span class="nc">testHammingDistance</span> <span class="o">{</span>
<span class="c1">//生成1亿个随机32位的Int类型数</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="kt">int</span><span class="o">[]</span> <span class="nf">randomInt</span><span class="o">(</span><span class="kt">int</span> <span class="n">num</span><span class="o">,</span> <span class="kt">int</span> <span class="n">seed</span><span class="o">)</span> <span class="o">{</span>
<span class="kt">int</span><span class="o">[]</span> <span class="n">a</span> <span class="o">=</span> <span class="k">new</span> <span class="kt">int</span><span class="o">[</span><span class="n">num</span><span class="o">];</span>
<span class="nc">Random</span> <span class="n">r</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Random</span><span class="o">(</span><span class="n">seed</span><span class="o">);</span>
<span class="nc">Arrays</span><span class="o">.</span><span class="na">setAll</span><span class="o">(</span><span class="n">a</span><span class="o">,</span> <span class="n">w</span> <span class="o">-></span> <span class="n">r</span><span class="o">.</span><span class="na">nextInt</span><span class="o">());</span>
<span class="k">return</span> <span class="n">a</span><span class="o">;</span>
<span class="o">}</span>
<span class="c1">//调用Java bitCount 求汉明距离</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="kt">int</span> <span class="nf">hammingDistance1</span><span class="o">(</span><span class="kt">int</span> <span class="n">hash1</span><span class="o">,</span> <span class="kt">int</span> <span class="n">hash2</span><span class="o">)</span> <span class="o">{</span>
<span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="n">hash1</span> <span class="o">^</span> <span class="n">hash2</span><span class="o">;</span>
<span class="k">return</span> <span class="nc">Integer</span><span class="o">.</span><span class="na">bitCount</span><span class="o">(</span><span class="n">i</span><span class="o">);</span>
<span class="o">}</span>
<span class="c1">//求解正数的二进制汉明距离</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="kt">int</span> <span class="nf">hammingDistance2</span><span class="o">(</span><span class="kt">int</span> <span class="n">hash1</span><span class="o">,</span> <span class="kt">int</span> <span class="n">hash2</span><span class="o">){</span>
<span class="kt">int</span> <span class="n">num</span> <span class="o">=</span> <span class="n">hash1</span> <span class="o">^</span> <span class="n">hash2</span><span class="o">;</span>
<span class="kt">int</span> <span class="n">count</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span>
<span class="k">for</span><span class="o">(;</span> <span class="n">num</span> <span class="o">></span> <span class="mi">0</span><span class="o">;</span> <span class="n">count</span><span class="o">++)</span>
<span class="o">{</span>
<span class="n">num</span> <span class="o">&=</span> <span class="o">(</span><span class="n">num</span> <span class="o">-</span> <span class="mi">1</span><span class="o">);</span>
<span class="o">}</span>
<span class="k">return</span> <span class="n">count</span><span class="o">;</span>
<span class="o">}</span>
<span class="c1">//主函数测试1亿对</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="kt">void</span> <span class="nf">main</span><span class="o">(</span><span class="nc">String</span><span class="o">[]</span> <span class="n">args</span><span class="o">)</span> <span class="o">{</span>
<span class="kt">int</span> <span class="n">tem</span> <span class="o">=</span> <span class="mi">4324523</span><span class="o">;</span>
<span class="kt">int</span><span class="o">[]</span> <span class="n">all</span> <span class="o">=</span> <span class="n">randomInt</span><span class="o">(</span><span class="mi">100000000</span><span class="o">,</span><span class="mi">123</span><span class="o">);</span>
<span class="kt">long</span> <span class="n">startTime</span> <span class="o">=</span> <span class="nc">System</span><span class="o">.</span><span class="na">currentTimeMillis</span><span class="o">();</span>
<span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="nl">x:</span><span class="n">all</span><span class="o">){</span>
<span class="kt">int</span> <span class="n">tem1</span> <span class="o">=</span> <span class="n">hammingDistance1</span><span class="o">(</span><span class="n">tem</span><span class="o">,</span><span class="n">x</span><span class="o">);</span>
<span class="o">}</span>
<span class="kt">long</span> <span class="n">endTime</span> <span class="o">=</span> <span class="nc">System</span><span class="o">.</span><span class="na">currentTimeMillis</span><span class="o">();</span>
<span class="kt">long</span> <span class="n">startTime2</span> <span class="o">=</span> <span class="nc">System</span><span class="o">.</span><span class="na">currentTimeMillis</span><span class="o">();</span>
<span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="nl">x:</span><span class="n">all</span><span class="o">){</span>
<span class="kt">int</span> <span class="n">tem2</span> <span class="o">=</span> <span class="n">hammingDistance2</span><span class="o">(</span><span class="n">tem</span><span class="o">,</span><span class="n">x</span><span class="o">);</span>
<span class="o">}</span>
<span class="kt">long</span> <span class="n">endTime2</span> <span class="o">=</span> <span class="nc">System</span><span class="o">.</span><span class="na">currentTimeMillis</span><span class="o">();</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"hammingDistance1 time used: "</span> <span class="o">+</span> <span class="o">(</span><span class="n">endTime</span><span class="o">-</span><span class="n">startTime</span><span class="o">)</span> <span class="o">+</span> <span class="s">"ms"</span><span class="o">);</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"hammingDistance2 time used: "</span> <span class="o">+</span> <span class="o">(</span><span class="n">endTime2</span><span class="o">-</span><span class="n">startTime2</span><span class="o">)</span> <span class="o">+</span> <span class="s">"ms"</span><span class="o">);</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>结果:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
hammingDistance1 <span class="nb">time </span>used: 5ms
hammingDistance2 <span class="nb">time </span>used: 980ms
Process finished with <span class="nb">exit </span>code 0
</code></pre></div></div>
<h1 id="一般算法">一般算法</h1>
<p>求汉明距离的核心是在数两个二进制数求<code class="language-plaintext highlighter-rouge">&</code>运算后1的个数,求1的个数最容易想到的是循环整个二进制获得1的个数。如下算法:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="kd">public</span> <span class="kt">int</span> <span class="nf">numberOf1</span><span class="o">(</span><span class="kt">int</span> <span class="n">hash1</span><span class="o">,</span> <span class="kt">int</span> <span class="n">hash2</span><span class="o">){</span>
<span class="kt">int</span> <span class="n">num</span> <span class="o">=</span> <span class="n">hash1</span> <span class="o">^</span> <span class="n">hash2</span><span class="o">;</span>
<span class="kt">int</span> <span class="n">count</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span>
<span class="k">while</span><span class="o">(</span><span class="n">n</span><span class="o">!=</span><span class="mi">0</span><span class="o">){</span>
<span class="k">if</span><span class="o">((</span><span class="n">num</span> <span class="o">&</span> <span class="mi">1</span><span class="o">)</span> <span class="o">==</span> <span class="mi">1</span><span class="o">){</span>
<span class="o">++</span><span class="n">count</span><span class="o">;</span>
<span class="o">}</span>
<span class="n">n</span> <span class="o">=</span> <span class="n">n</span> <span class="o">>></span> <span class="mi">1</span><span class="o">;</span>
<span class="o">}</span>
<span class="k">return</span> <span class="n">count</span><span class="o">;</span>
</code></pre></div></div>
<ul>
<li>这种方法很容易理解,就是每次循环的时候对1取<code class="language-plaintext highlighter-rouge">&</code>操作,若结果等于1就统计出了一个1,然后每次循环后右移<code class="language-plaintext highlighter-rouge">>></code>使高位的0或1移动到低位。这种算法的
时间复杂度为<code class="language-plaintext highlighter-rouge">O(n)</code>,其中n表示bit的位数。</li>
</ul>
<h1 id="优化算法">优化算法</h1>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="kd">public</span> <span class="kt">int</span> <span class="nf">hammingDistance2</span><span class="o">(</span><span class="kt">int</span> <span class="n">hash1</span><span class="o">,</span> <span class="kt">int</span> <span class="n">hash2</span><span class="o">){</span>
<span class="kt">int</span> <span class="n">num</span> <span class="o">=</span> <span class="n">hash1</span> <span class="o">^</span> <span class="n">hash2</span><span class="o">;</span>
<span class="kt">int</span> <span class="n">count</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span>
<span class="k">for</span><span class="o">(;</span> <span class="n">num</span> <span class="o">></span> <span class="mi">0</span><span class="o">;</span> <span class="n">count</span><span class="o">++)</span>
<span class="o">{</span>
<span class="n">num</span> <span class="o">&=</span> <span class="o">(</span><span class="n">num</span> <span class="o">-</span> <span class="mi">1</span><span class="o">);</span>
<span class="o">}</span>
<span class="k">return</span> <span class="n">count</span><span class="o">;</span>
<span class="o">}</span>
</code></pre></div></div>
<ul>
<li>优化后的方法通过二进制的运算法则进行优化,优化后的时间复杂度仍然为<code class="language-plaintext highlighter-rouge">O(n)</code>,但是这里的n代表着1的个数最差的结果n才代表bit位,比第一个算法n代表bit位要好。
二进制的运算原则是逢二进一,每次循环时候我都让num减去1通过跟自身&操作消除低位1,最后num<0循环停止。有多少个1就循环多少次。</li>
</ul>
<p>例如,二进制<code class="language-plaintext highlighter-rouge">0b1110</code>统计为3个1的过程如下:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="c1">// 第一次迭代:</span>
<span class="mb">0b1110</span> <span class="o">-</span> <span class="mi">1</span> <span class="o">=</span> <span class="mb">0b1110</span> <span class="o">-</span> <span class="mb">0b0001</span> <span class="o">=</span> <span class="mb">0b1101</span>
<span class="mb">0b1110</span> <span class="o">&</span> <span class="mb">0b1101</span> <span class="o">=</span> <span class="mb">0b1100</span>
<span class="c1">// 第二次迭代:</span>
<span class="mb">0b1100</span> <span class="o">-</span> <span class="mi">1</span> <span class="o">=</span> <span class="mb">0b1100</span> <span class="o">-</span> <span class="mb">0b0001</span> <span class="o">=</span> <span class="mb">0b1011</span>
<span class="mb">0b1100</span> <span class="o">&</span> <span class="mb">0b1011</span> <span class="o">=</span> <span class="mb">0b1000</span>
<span class="c1">// 第三次迭代:</span>
<span class="mb">0b1000</span> <span class="o">-</span> <span class="mi">1</span> <span class="o">=</span> <span class="mb">0b1000</span> <span class="o">-</span> <span class="mb">0b0001</span> <span class="o">=</span> <span class="mb">0b0111</span>
<span class="mb">0b1000</span> <span class="o">&</span> <span class="mb">0b0111</span> <span class="o">=</span> <span class="mb">0b0000</span>
</code></pre></div></div>
<h1 id="java源码bitcount算法">Java源码bitCount算法</h1>
<ul>
<li>Java的32位二进制Integer.bitCount源码如下:</li>
</ul>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="kd">public</span> <span class="kd">static</span> <span class="kt">int</span> <span class="nf">bitCount</span><span class="o">(</span><span class="kt">int</span> <span class="n">i</span><span class="o">)</span> <span class="o">{</span>
<span class="n">i</span> <span class="o">=</span> <span class="n">i</span> <span class="o">-</span> <span class="o">((</span><span class="n">i</span> <span class="o">>>></span> <span class="mi">1</span><span class="o">)</span> <span class="o">&</span> <span class="mh">0x55555555</span><span class="o">);</span>
<span class="n">i</span> <span class="o">=</span> <span class="o">(</span><span class="n">i</span> <span class="o">&</span> <span class="mh">0x33333333</span><span class="o">)</span> <span class="o">+</span> <span class="o">((</span><span class="n">i</span> <span class="o">>>></span> <span class="mi">2</span><span class="o">)</span> <span class="o">&</span> <span class="mh">0x33333333</span><span class="o">);</span>
<span class="n">i</span> <span class="o">=</span> <span class="o">(</span><span class="n">i</span> <span class="o">+</span> <span class="o">(</span><span class="n">i</span> <span class="o">>>></span> <span class="mi">4</span><span class="o">))</span> <span class="o">&</span> <span class="mh">0x0f0f0f0f</span><span class="o">;</span>
<span class="n">i</span> <span class="o">=</span> <span class="n">i</span> <span class="o">+</span> <span class="o">(</span><span class="n">i</span> <span class="o">>>></span> <span class="mi">8</span><span class="o">);</span>
<span class="n">i</span> <span class="o">=</span> <span class="n">i</span> <span class="o">+</span> <span class="o">(</span><span class="n">i</span> <span class="o">>>></span> <span class="mi">16</span><span class="o">);</span>
<span class="k">return</span> <span class="n">i</span> <span class="o">&</span> <span class="mh">0x3f</span><span class="o">;</span>
<span class="o">}</span>
</code></pre></div></div>
<h2 id="java源码实现了复杂度为o1的算法">Java源码实现了复杂度为O(1)的算法</h2>
<ul>
<li>当我们想知道一个二进制数中有多少个1时候最容易想到的方法就如前两种方法一个一个的数,数一下一共多少个1就完事了。但是Java源码使用的方法是
先两个一对儿数一下有多少个1,存到原来的位置。然后对上面的数的结果再成对统计以此类推。</li>
</ul>
<p>以8位为例,算法思想如下:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
二进制 十进制<span class="o">(</span>count个数<span class="o">)</span>
1 0 1 1 1 1 0 1 1 0 1 1 1 1 0 1
<span class="se">\/</span> <span class="se">\/</span> <span class="se">\/</span> <span class="se">\/</span> <span class="se">\/</span> <span class="se">\/</span> <span class="se">\/</span> <span class="se">\/</span>
01 10 10 01 1 2 2 1
<span class="se">\ </span> / <span class="se">\ </span> / <span class="se">\ </span> / <span class="se">\ </span> /
0011 0011 3 3
<span class="se">\ </span> / <span class="se">\ </span> /
0110 6
</code></pre></div></div>
<p>这种count方法就如一个倒立的二叉树一样。每一层的节点个数为2^(n-1)个,也就是说对于32位的二进制2^(n-1)=32,n=5。所以,我们只需要数5次,对于64位二进制只需要
数6次就能知道二进制中有多少个1了,这种方法实现了时间复杂度为O(1)优秀算法。</p>
<h2 id="java源码算法实现机理">Java源码算法实现机理</h2>
<ul>
<li>Java源码中每次<code class="language-plaintext highlighter-rouge">&</code>运算各个数的二进制,主要是为了抹除成对统计时候的左位(进行右移位运算时候高位对低位产生的影响)。</li>
</ul>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="mh">0x55555555</span> <span class="err"></span><span class="mb">0b01010101010101010101010101010101</span><span class="err"></span>
<span class="mh">0x33333333</span> <span class="err"></span><span class="mb">0b00110011001100110011001100110011</span><span class="err"></span>
<span class="mh">0x0f0f0f0f</span> <span class="err"></span><span class="mb">0b00001111000011110000111100001111</span><span class="err"></span>
<span class="mh">0x00ff00ff</span> <span class="mb">0b00000000111111110000000011111111</span>
<span class="mh">0x0000ffff</span> <span class="err"></span><span class="mb">0b00000000000000001111111111111111</span><span class="err"></span>
<span class="mh">0x3f</span> <span class="err"></span><span class="mb">0b00111111</span><span class="err"></span>
</code></pre></div></div>
<ul>
<li>算法实现</li>
</ul>
<blockquote>
<p>i & 0x55555555实现整数i抹除左一位,然后错位相加。
(i »> 1) & 0x55555555表示:左位移到右边,再把左位抹除,这样就可以计算两个bit位上1的个数了:0b1011=>0b0001 + 0b0101 = 0b0110左两位有1个1,右两位有2个1。
这时i中存储了每两位的统计结果,可以进行两两相加,最后求和</p>
</blockquote>
<blockquote>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="mb">0b11</span> <span class="mi">11</span> <span class="mi">11</span> <span class="mi">11</span> <span class="mi">11</span> <span class="o">(</span><span class="n">i</span> <span class="o">&</span> <span class="mh">0x55555555</span><span class="o">)</span> <span class="o">+</span> <span class="o">((</span><span class="n">i</span> <span class="o">>>></span> <span class="mi">1</span><span class="o">)</span> <span class="o">&</span> <span class="mh">0x55555555</span><span class="o">)</span> <span class="o">=</span> <span class="mb">0b0101010101</span><span class="err"></span> <span class="o">+</span> <span class="mb">0b0101010101</span> <span class="o">=</span> <span class="mb">0b1010101010</span>
<span class="mb">0b10</span> <span class="mi">10</span> <span class="mi">10</span> <span class="mi">10</span> <span class="mi">10</span> <span class="o">(</span><span class="n">i</span> <span class="o">&</span> <span class="mh">0x33333333</span><span class="o">)</span> <span class="o">+</span> <span class="o">((</span><span class="n">i</span> <span class="o">>>></span> <span class="mi">2</span><span class="o">)</span> <span class="o">&</span> <span class="mh">0x33333333</span><span class="o">)</span> <span class="o">=</span> <span class="mb">0b1000100010</span> <span class="o">+</span> <span class="mb">0b00100010</span> <span class="o">=</span> <span class="mb">0b1001000100</span>
<span class="mb">0b10</span> <span class="mo">01</span> <span class="mo">00</span> <span class="mo">01</span> <span class="mo">00</span> <span class="o">(</span><span class="n">i</span> <span class="o">&</span> <span class="mh">0x0f0f0f0f</span><span class="o">)</span> <span class="o">+</span> <span class="o">((</span><span class="n">i</span> <span class="o">>>></span> <span class="mi">4</span><span class="o">)</span> <span class="o">&</span> <span class="mh">0x0f0f0f0f</span><span class="o">)</span> <span class="o">=</span> <span class="mb">0b1000000100</span> <span class="o">+</span> <span class="mb">0b0100</span> <span class="o">=</span> <span class="mb">0b1000001000</span>
<span class="mb">0b10</span> <span class="mo">00</span> <span class="mo">00</span> <span class="mi">10</span> <span class="mo">00</span> <span class="o">(</span><span class="n">i</span> <span class="o">&</span> <span class="mh">0x00ff00ff</span><span class="o">)</span> <span class="o">+</span> <span class="o">((</span><span class="n">i</span> <span class="o">>>></span> <span class="mi">8</span><span class="o">)</span> <span class="o">&</span> <span class="mh">0x00ff00ff</span><span class="o">)</span> <span class="o">=</span> <span class="mb">0b1000</span> <span class="o">+</span> <span class="mb">0b10</span> <span class="o">=</span> <span class="mb">0b1010</span>
<span class="mb">0b00</span> <span class="mo">00</span> <span class="mo">00</span> <span class="mi">10</span> <span class="mi">10</span> <span class="o">(</span><span class="n">i</span> <span class="o">&</span> <span class="mh">0x0000ffff</span><span class="o">)</span> <span class="o">+</span> <span class="o">((</span><span class="n">i</span> <span class="o">>>></span> <span class="mi">16</span><span class="o">)</span> <span class="o">&</span> <span class="mh">0x0000ffff</span><span class="o">)</span> <span class="o">=</span> <span class="mb">0b1010</span> <span class="o">+</span> <span class="mi">0</span> <span class="o">=</span> <span class="mb">0b1010</span>
<span class="n">dec</span> <span class="mi">10</span>
</code></pre></div> </div>
</blockquote>
<ul>
<li>算法原型</li>
</ul>
<blockquote>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">static</span> <span class="kt">int</span> <span class="nf">bitCount</span><span class="o">(</span><span class="kt">int</span> <span class="n">i</span><span class="o">)</span> <span class="o">{</span>
<span class="n">i</span> <span class="o">=</span> <span class="o">(</span><span class="n">i</span> <span class="o">&</span> <span class="mh">0x55555555</span><span class="o">)</span> <span class="o">+</span> <span class="o">((</span><span class="n">i</span> <span class="o">>>></span> <span class="mi">1</span><span class="o">)</span> <span class="o">&</span> <span class="mh">0x55555555</span><span class="o">);</span>
<span class="n">i</span> <span class="o">=</span> <span class="o">(</span><span class="n">i</span> <span class="o">&</span> <span class="mh">0x33333333</span><span class="o">)</span> <span class="o">+</span> <span class="o">((</span><span class="n">i</span> <span class="o">>>></span> <span class="mi">2</span><span class="o">)</span> <span class="o">&</span> <span class="mh">0x33333333</span><span class="o">);</span>
<span class="n">i</span> <span class="o">=</span> <span class="o">(</span><span class="n">i</span> <span class="o">&</span> <span class="mh">0x0f0f0f0f</span><span class="o">)</span> <span class="o">+</span> <span class="o">((</span><span class="n">i</span> <span class="o">>>></span> <span class="mi">4</span><span class="o">)</span> <span class="o">&</span> <span class="mh">0x0f0f0f0f</span><span class="o">);</span>
<span class="n">i</span> <span class="o">=</span> <span class="o">(</span><span class="n">i</span> <span class="o">&</span> <span class="mh">0x00ff00ff</span><span class="o">)</span> <span class="o">+</span> <span class="o">((</span><span class="n">i</span> <span class="o">>>></span> <span class="mi">8</span><span class="o">)</span> <span class="o">&</span> <span class="mh">0x00ff00ff</span><span class="o">);</span>
<span class="n">i</span> <span class="o">=</span> <span class="o">(</span><span class="n">i</span> <span class="o">&</span> <span class="mh">0x0000ffff</span><span class="o">)</span> <span class="o">+</span> <span class="o">((</span><span class="n">i</span> <span class="o">>>></span> <span class="mi">16</span><span class="o">)</span> <span class="o">&</span> <span class="mh">0x0000ffff</span><span class="o">);</span>
<span class="k">return</span> <span class="n">i</span><span class="o">;</span>
<span class="o">}</span>
</code></pre></div> </div>
</blockquote>
<ul>
<li>原型优化</li>
</ul>
<p>1.算法第一步可以优化为:i = i - ((i »> 1) & 0x55555555); 减少一次位运算。原理如下:</p>
<ul>
<li>对于两位二进制有如下规律:</li>
</ul>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
00 <span class="o">=</span> 00 - 00<span class="p">;</span>
01 <span class="o">=</span> 01 - 00<span class="p">;</span>
01 <span class="o">=</span> 10 - 01<span class="p">;</span>
10 <span class="o">=</span> 11 - 01<span class="p">;</span>
</code></pre></div></div>
<p>所以,在统计两位中1的个数的时候可以由算法i = i - (i»>1)得到,如果大于两位,为了避免高位右移时候对低位产生的影响,可以通过对01进行<code class="language-plaintext highlighter-rouge">&</code>运算
的方法抹除这种影响。</p>
<p>两位<code class="language-plaintext highlighter-rouge">&</code>运算如下:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
i <span class="o">=</span> i - <span class="o">(</span>i>>>1&01<span class="o">)</span>
</code></pre></div></div>
<p>推广到32位:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
i <span class="o">=</span> i - <span class="o">((</span>i <span class="o">>>></span> 1<span class="o">)</span> & 0x55555555<span class="o">)</span>
</code></pre></div></div>
<p>2.算法第二步没有优化方法</p>
<p>3.算法第三步:实际是计算每个byte中的1的数量,最多8(0b1000)个,占4bit,可以最后进行位与运算消位,减少一次&运算:i = (i + (i »> 4)) & 0x0f0f0f0f</p>
<p>4.第四,五步:同上理由,可以最后消位。但是由于int最多32(0b100000)个1,所以这两步可以不消位,最后一步把不需要的bit位抹除就可以了:i & 0x3f</p>
<p>最后得到Java源码最简洁的写法。</p>
<h1 id="java位运算符">Java位运算符</h1>
<table>
<thead>
<tr>
<th>操作符</th>
<th>描述</th>
</tr>
</thead>
<tbody>
<tr>
<td>&</td>
<td>如果相对应位都是1,则结果为1,否则为0</td>
</tr>
<tr>
<td>|</td>
<td>如果相对应位都是0,则结果为0,否则为1</td>
</tr>
<tr>
<td>^</td>
<td>如果相对应位值相同,则结果为0,否则为1</td>
</tr>
<tr>
<td>〜</td>
<td>按位取反运算符翻转操作数的每一位,即0变成1,1变成0。</td>
</tr>
<tr>
<td>«</td>
<td>按位左移运算符。各二进制位全部左移 若干位,高位丢弃,低位补0</td>
</tr>
<tr>
<td>»</td>
<td>按位右移运算符。各二进制位全部右移 若干位,高位补符号位</td>
</tr>
<tr>
<td>»></td>
<td>按位右移补零操作符。左操作数的值按右操作数指定的位数右移,移动得到的空位以零填充。</td>
</tr>
</tbody>
</table>
<ul>
<li>参考:</li>
</ul>
<p><a href="https://segmentfault.com/a/1190000015763941?utm_source=tag-newest">java源码Integer.bitCount算法解析</a></p>Yang Fang汉明距离(Hamming Distance)的基本思想很简单,就是找不同。当求由01二进制组成 的向量间的汉明距离可以由位运算直接进行,速度非常快。好的算法能让计算速度达到极快,Java内置bitCount源码就实现了一种速度极快的算法。在Linux机器 (CPU: i7-4790 @ 3.6GHz)测试了1亿对汉明距离只用0.5ms,是普通算法的196倍。Java 创建销毁对象——静态工厂方法2019-02-27T00:00:00+00:002019-02-27T00:00:00+00:00https://yangfangs.github.io//2019/02/27/use-static-factory-method<p>什么是静态工厂方法?在 Java 中,获得一个类实例常常使用 new 关键字,通过构造函数来实现对象的创建。不通过 new,
而是用一个静态方法来对外提供自身实例的方法,即为<strong>静态工厂方法(Static factory method)</strong>。</p>
<h1 id="使用静态工厂方法的优势">使用静态工厂方法的优势</h1>
<ul>
<li>Effective Java中有如下总结:
<ol>
<li>静态工厂方法与构造器不同的第一优势在于,它们有名字。</li>
<li>不必在每次被调用时都创建新对象。</li>
<li>可以返回原返回类型的任何子类对象。</li>
<li>在创建参数化类型实例的时候,能使代码变得简洁。</li>
</ol>
</li>
</ul>
<h1 id="1静态工厂方法与构造器不同的第一优势在于它们有名字">1.静态工厂方法与构造器不同的第一优势在于,它们有名字</h1>
<ul>
<li>例如:Boolean类的一个构造方法,以及通过该构造方法创建一个Boolean对象;</li>
</ul>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="kd">public</span> <span class="nf">Boolean</span><span class="o">(</span><span class="nc">String</span> <span class="n">s</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">(</span><span class="n">toBoolean</span><span class="o">(</span><span class="n">s</span><span class="o">));</span>
<span class="o">}</span>
<span class="nc">Boolean</span> <span class="n">bTrue</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Boolean</span><span class="o">(</span><span class="s">"true"</span><span class="o">);</span>
</code></pre></div></div>
<ul>
<li>使用静态工厂方法可以如下</li>
</ul>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="kd">public</span> <span class="kd">static</span> <span class="nc">Boolean</span> <span class="nf">valueOf</span><span class="o">(</span><span class="nc">String</span> <span class="n">s</span><span class="o">)</span> <span class="o">{</span>
<span class="k">return</span> <span class="nf">toBoolean</span><span class="o">(</span><span class="n">s</span><span class="o">)</span> <span class="o">?</span> <span class="no">TRUE</span> <span class="o">:</span> <span class="no">FALSE</span><span class="o">;</span>
<span class="o">}</span>
<span class="nc">Boolean</span> <span class="n">bTrue</span> <span class="o">=</span> <span class="nc">Boolean</span><span class="o">.</span><span class="na">valueOf</span><span class="o">(</span><span class="s">"true"</span><span class="o">);</span>
</code></pre></div></div>
<p>在这里我们使用 valueOf 在实例化一个对象,并没有使用 new 关键字,同时实例化时候使用了新的名称 <code class="language-plaintext highlighter-rouge">valueOf</code> 而不是 <code class="language-plaintext highlighter-rouge">Boolean</code>构造器的名称。</p>
<h1 id="2不必在每次被调用时都创建新对象">2.不必在每次被调用时都创建新对象</h1>
<ul>
<li>
<p>JDK中的Boolean类的valueOf方法就是使用这个优势,在Boolean类中,有两个事先创建好的Boolean对象(True,False)(静态类的原因),并不是每次调用时候在重新创建,增加 JVM 开销</p>
</li>
<li>
<p>例如:</p>
</li>
</ul>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="kd">public</span> <span class="kd">final</span> <span class="kd">class</span> <span class="nc">Boolean</span> <span class="kd">implements</span> <span class="n">java</span><span class="o">.</span><span class="na">io</span><span class="o">.</span><span class="na">Serializable</span><span class="o">,</span>
<span class="nc">Comparable</span><span class="o"><</span><span class="nc">Boolean</span><span class="o">></span>
<span class="o">{</span>
<span class="cm">/**
* The {@code Boolean} object corresponding to the primitive
* value {@code true}.
*/</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="kd">final</span> <span class="nc">Boolean</span> <span class="no">TRUE</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Boolean</span><span class="o">(</span><span class="kc">true</span><span class="o">);</span>
<span class="cm">/**
* The {@code Boolean} object corresponding to the primitive
* value {@code false}.
*/</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="kd">final</span> <span class="nc">Boolean</span> <span class="no">FALSE</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Boolean</span><span class="o">(</span><span class="kc">false</span><span class="o">);</span>
</code></pre></div></div>
<ul>
<li>使用Boolean.valueOf(“true”)静态工厂方法调用时候返回的就是这两个实例的引用。这样可以避免每次调用都自重复创建对象。</li>
</ul>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="kd">public</span> <span class="kd">static</span> <span class="nc">Boolean</span> <span class="nf">valueOf</span><span class="o">(</span><span class="nc">String</span> <span class="n">s</span><span class="o">)</span> <span class="o">{</span>
<span class="k">return</span> <span class="nf">toBoolean</span><span class="o">(</span><span class="n">s</span><span class="o">)</span> <span class="o">?</span> <span class="no">TRUE</span> <span class="o">:</span> <span class="no">FALSE</span><span class="o">;</span>
<span class="o">}</span>
</code></pre></div></div>
<h1 id="3可以返回原返回类型的任何子类对象">3.可以返回原返回类型的任何子类对象。</h1>
<ul>
<li>
<p>我们在选择返回对象的类时,使用静态工厂方法就更有灵活性。这种灵活性的一种应用是API可以返回对象,同时又不会使对象的类变成公有的。这种方式隐藏实现类会使API变得非常简洁。
对于构造方法只能返回确切的自身类型,而静态工厂方法则能够更加灵活,可以根据需要方便地返回任何它的子类型的实例。</p>
</li>
<li>
<p>例如:</p>
</li>
</ul>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="nc">Class</span> <span class="nc">Person</span> <span class="o">{</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="nc">Person</span> <span class="nf">getInstance</span><span class="o">(){</span>
<span class="c1">//直接返回子类</span>
<span class="k">return</span> <span class="k">new</span> <span class="nf">Player</span><span class="o">()</span>
<span class="c1">// return new Cooker()</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="c1">// 子类</span>
<span class="nc">Class</span> <span class="nc">Player</span> <span class="kd">extends</span> <span class="nc">Person</span><span class="o">{</span>
<span class="o">}</span>
<span class="c1">// 子类</span>
<span class="nc">Class</span> <span class="nc">Cooker</span> <span class="kd">extends</span> <span class="nc">Person</span><span class="o">{</span>
<span class="o">}</span>
</code></pre></div></div>
<h1 id="4在创建参数化类型实例的时候能使代码变得简洁">4.在创建参数化类型实例的时候,能使代码变得简洁。</h1>
<ul>
<li>
<p>主要对与泛型书写简化:</p>
</li>
<li>
<p>例如:</p>
</li>
</ul>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="nc">Map</span><span class="o"><</span><span class="nc">String</span><span class="o">,</span><span class="nc">Date</span><span class="o">></span> <span class="n">map</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">HashMap</span><span class="o"><</span><span class="nc">String</span><span class="o">,</span><span class="nc">Date</span><span class="o">>();</span>
<span class="c1">//更加简洁,不需要重复指明类型参数,可以自行推导出来</span>
<span class="nc">Map</span><span class="o"><</span><span class="nc">String</span><span class="o">,</span><span class="nc">Date</span><span class="o">></span> <span class="n">map</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">HashMap</span><span class="o">.</span><span class="na">newInstance</span><span class="o">();</span>
</code></pre></div></div>
<p>Java7以后的版本对于一个已知类型的变量进行赋值时,由于泛型参数是可以被推导出,所以可以在创建实例时省略掉泛型参数。所以这个优势实际上以及不存在了。</p>
<h1 id="静态工厂方法额外的用处">静态工厂方法额外的用处</h1>
<h2 id="1可以有多个参数相同但名称不同的工厂方法">1.可以有多个参数相同但名称不同的工厂方法</h2>
<ul>
<li>例如:</li>
</ul>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">class</span> <span class="nc">Child</span><span class="o">{</span>
<span class="kt">int</span> <span class="n">age</span> <span class="o">=</span> <span class="mi">10</span><span class="o">;</span>
<span class="kt">int</span> <span class="n">weight</span> <span class="o">=</span> <span class="mi">30</span><span class="o">;</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="nc">Child</span> <span class="nf">newChild</span><span class="o">(</span><span class="kt">int</span> <span class="n">age</span><span class="o">,</span> <span class="kt">int</span> <span class="n">weight</span><span class="o">)</span> <span class="o">{</span>
<span class="nc">Child</span> <span class="n">child</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Child</span><span class="o">();</span>
<span class="n">child</span><span class="o">.</span><span class="na">weight</span> <span class="o">=</span> <span class="n">weight</span><span class="o">;</span>
<span class="n">child</span><span class="o">.</span><span class="na">age</span> <span class="o">=</span> <span class="n">age</span><span class="o">;</span>
<span class="k">return</span> <span class="n">child</span><span class="o">;</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="nc">Child</span> <span class="nf">newChildWithWeight</span><span class="o">(</span><span class="kt">int</span> <span class="n">weight</span><span class="o">)</span> <span class="o">{</span>
<span class="nc">Child</span> <span class="n">child</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Child</span><span class="o">();</span>
<span class="n">child</span><span class="o">.</span><span class="na">weight</span> <span class="o">=</span> <span class="n">weight</span><span class="o">;</span>
<span class="k">return</span> <span class="n">child</span><span class="o">;</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="nc">Child</span> <span class="nf">newChildWithAge</span><span class="o">(</span><span class="kt">int</span> <span class="n">age</span><span class="o">)</span> <span class="o">{</span>
<span class="nc">Child</span> <span class="n">child</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Child</span><span class="o">();</span>
<span class="n">child</span><span class="o">.</span><span class="na">age</span> <span class="o">=</span> <span class="n">age</span><span class="o">;</span>
<span class="k">return</span> <span class="n">child</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<h2 id="2可以减少对外暴露的属性">2.可以减少对外暴露的属性</h2>
<ul>
<li>
<p>这种方法也常常使用枚举来代替常量值来设置,当然如果不想用枚举的话,静态工厂方法也是一个很好的办法。</p>
</li>
<li>
<p>例如:</p>
</li>
</ul>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="kd">class</span> <span class="nc">Player</span> <span class="o">{</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="kd">final</span> <span class="kt">int</span> <span class="no">TYPE_RUNNER</span> <span class="o">=</span> <span class="mi">1</span><span class="o">;</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="kd">final</span> <span class="kt">int</span> <span class="no">TYPE_SWIMMER</span> <span class="o">=</span> <span class="mi">2</span><span class="o">;</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="kd">final</span> <span class="kt">int</span> <span class="no">TYPE_RACER</span> <span class="o">=</span> <span class="mi">3</span><span class="o">;</span>
<span class="kt">int</span> <span class="n">type</span><span class="o">;</span>
<span class="kd">private</span> <span class="nf">Player</span><span class="o">(</span><span class="kt">int</span> <span class="n">type</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">type</span> <span class="o">=</span> <span class="n">type</span><span class="o">;</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="nc">Player</span> <span class="nf">newRunner</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="k">new</span> <span class="nf">Player</span><span class="o">(</span><span class="no">TYPE_RUNNER</span><span class="o">);</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="nc">Player</span> <span class="nf">newSwimmer</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="k">new</span> <span class="nf">Player</span><span class="o">(</span><span class="no">TYPE_SWIMMER</span><span class="o">);</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="nc">Player</span> <span class="nf">newRacer</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="k">new</span> <span class="nf">Player</span><span class="o">(</span><span class="no">TYPE_RACER</span><span class="o">);</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<h1 id="参考">参考:</h1>
<p><a href="https://book.douban.com/subject/3360807/">Effective java 中文版(第2版)</a></p>
<p><a href="https://www.jianshu.com/p/ceb5ec8f1174">关于 Java 的静态工厂方法,看这一篇就够了</a></p>Yang Fang什么是静态工厂方法?在 Java 中,获得一个类实例常常使用 new 关键字,通过构造函数来实现对象的创建。不通过 new, 而是用一个静态方法来对外提供自身实例的方法,即为静态工厂方法(Static factory method)。Python 快速打包发布软件PyPi上2018-08-06T00:00:00+00:002018-08-06T00:00:00+00:00https://yangfangs.github.io//2018/08/06/python-distribution-packages<p><a href="https://pypi.org/">PyPi</a>是一个python包的仓库,就如 R 的<a href="https://cran.r-project.org/mirror-howto.html">CRAN</a>和 perl 的<a href="https://www.perl.org/cpan.html">CPAN</a>一样,里面有很多别人打包发布的python包,你可以通过easy_install或者pip进行安装,方便用户更方面的使用你的代码模块。
本文记录了如何创建自己的 Python 包,以及打包发布到PyPi上。</p>
<h1 id="首先安装打包必要的工具">首先安装打包必要的工具</h1>
<ul>
<li>打包必须依赖工具</li>
</ul>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>pip <span class="nb">install </span>setuptools
</code></pre></div></div>
<ul>
<li>方便上传工具</li>
</ul>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>pip <span class="nb">install </span>twine
</code></pre></div></div>
<h1 id="在pypi官网注册一个pypi个人账户如下">在<a href="https://pypi.org/">PyPi</a>官网注册一个PyPi个人账户如下</h1>
<p><img src="/images/posts/Python/PyPI_Index.png" alt="PyPi" /></p>
<h1 id="账户和密码本地配置">账户和密码本地配置</h1>
<p>可以配置到$HOME/.pypirc文件中,就不用多次输入了</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[pypi]
username = <username>
password = <password>
</code></pre></div></div>
<h1 id="配置本地打包配置文件setuppy">配置本地打包配置文件<code class="language-plaintext highlighter-rouge">setup.py</code></h1>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">setuptools</span> <span class="kn">import</span> <span class="n">setup</span><span class="p">,</span> <span class="n">find_packages</span>
<span class="kn">from</span> <span class="nn">version.version</span> <span class="kn">import</span> <span class="n">VERSION</span>
<span class="n">GFICLEE_VERSION</span> <span class="o">=</span> <span class="n">VERSION</span>
<span class="n">setup</span><span class="p">(</span>
<span class="n">name</span><span class="o">=</span><span class="s">'GFICLEE'</span><span class="p">,</span>
<span class="n">version</span><span class="o">=</span><span class="n">GFICLEE_VERSION</span><span class="p">,</span>
<span class="n">packages</span><span class="o">=</span><span class="n">find_packages</span><span class="p">(),</span>
<span class="n">entry_points</span><span class="o">=</span><span class="p">{</span>
<span class="s">"console_scripts"</span><span class="p">:</span> <span class="p">[</span><span class="s">'GFICLEE = predict.main:main'</span><span class="p">]</span>
<span class="p">},</span>
<span class="n">install_requires</span><span class="o">=</span><span class="p">[</span>
<span class="s">"numpy==1.14.3"</span><span class="p">,</span>
<span class="s">"pandas==0.22.0"</span><span class="p">,</span>
<span class="p">],</span>
<span class="n">url</span><span class="o">=</span><span class="s">'https://github.com/yangfangs/GFICLEE'</span><span class="p">,</span>
<span class="n">license</span><span class="o">=</span><span class="s">'GNU General Public License v3.0'</span><span class="p">,</span>
<span class="n">author</span><span class="o">=</span><span class="s">'Yang Fang'</span><span class="p">,</span>
<span class="n">author_email</span><span class="o">=</span><span class="s">'yangfangscu@gmail.com'</span><span class="p">,</span>
<span class="n">description</span><span class="o">=</span><span class="s">'Gene function inferred by common loss evolution events'</span>
<span class="p">)</span>
</code></pre></div></div>
<ul>
<li>这些是打包必要的说明对应如下:</li>
</ul>
<table>
<thead>
<tr>
<th>名称</th>
<th>说明</th>
</tr>
</thead>
<tbody>
<tr>
<td>name</td>
<td>项目名称</td>
</tr>
<tr>
<td>version</td>
<td>项目版本</td>
</tr>
<tr>
<td>packages</td>
<td>项目包含的数据</td>
</tr>
<tr>
<td>entry_points</td>
<td>项目的主入口</td>
</tr>
<tr>
<td>install_requires</td>
<td>项目依赖包</td>
</tr>
<tr>
<td>url</td>
<td>项目地址</td>
</tr>
<tr>
<td>author</td>
<td>项目作者</td>
</tr>
<tr>
<td>author_email</td>
<td>作者邮箱</td>
</tr>
<tr>
<td>description</td>
<td>项目简要描述</td>
</tr>
</tbody>
</table>
<blockquote>
<p>注: 项目版本一般不直接写入<code class="language-plaintext highlighter-rouge">setup.py</code>中,一般另写其他Python脚本中然后导入,便于统一管理和发布新版本。
<code class="language-plaintext highlighter-rouge">packages</code>可以使用 <code class="language-plaintext highlighter-rouge">find_packages()</code>函数实习本项目内自动寻找。</p>
</blockquote>
<h1 id="打包">打包</h1>
<ul>
<li>打包前可以先check一下</li>
</ul>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
python setup.py check
</code></pre></div></div>
<ul>
<li>执行打包命令</li>
</ul>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="n">python</span> <span class="n">setup</span><span class="p">.</span><span class="n">py</span> <span class="n">sdist</span>
</code></pre></div></div>
<blockquote>
<p>打包完成会生成相应的文件保存在 <code class="language-plaintext highlighter-rouge">dist/</code>目录下 名为<code class="language-plaintext highlighter-rouge">tar.gz</code>结尾文件</p>
</blockquote>
<h1 id="发布python-包">发布Python 包</h1>
<p>1.在Home新建一个隐藏文件名为<code class="language-plaintext highlighter-rouge">.pypirc</code>,写入PyPi账户以及密码信息,这样每次上传不需要再繁琐的输入用户名和密码了。</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">[</span>distutils]
index-servers <span class="o">=</span>
pypi
<span class="o">[</span>pypi]
username:your_username
password:your_password
</code></pre></div></div>
<p>2.本地测试</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>python setup.py <span class="nb">install</span>
</code></pre></div></div>
<p>3.注册包</p>
<ul>
<li>上传前需要注册一下包的名称,因为这个名称必须独一无二,如被占用则注册不通过。</li>
</ul>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>python setup.py register
</code></pre></div></div>
<p>4.检测是否符合Pypi要求</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
twine check dist/<span class="k">**</span>_.tar.gz
</code></pre></div></div>
<p>5.上传</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>twine upload dist/<span class="k">**</span>_.tar.gz
</code></pre></div></div>
<p>完成上传你就可以在Pypi上看到你上传的包了。并且可以使用<code class="language-plaintext highlighter-rouge">pip install</code>安装你的包了。</p>
<h1 id="使用markdown-格式的readme文件">使用markdown 格式的readme文件</h1>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
from setuptools import setup
# read the contents of your README file
from os import path
this_directory = path.abspath(path.dirname(__file__))
with open(path.join(this_directory, 'README.md'), encoding='utf-8') as f:
long_description = f.read()
setup(
name='an_example_package',
# other arguments omitted
long_description=long_description,
long_description_content_type='text/markdown'
)
</code></pre></div></div>Yang FangPyPi是一个python包的仓库,就如 R 的CRAN和 perl 的CPAN一样,里面有很多别人打包发布的python包,你可以通过easy_install或者pip进行安装,方便用户更方面的使用你的代码模块。 本文记录了如何创建自己的 Python 包,以及打包发布到PyPi上。似然(likelihood)与概率(probability)的区别2018-04-06T00:00:00+00:002018-04-06T00:00:00+00:00https://yangfangs.github.io//2018/04/06/the-different-of-likelihood-and-probability<p>很多时候基础的概念没搞懂,或者知道了只知道照着例子套用,知其然而不知其所以然。对于知识的理解只能到达会运用的层面。而对于知识的创新运用是远远不足的。</p>
<p>似然和概率在统计学中是经常见到的两个术语,有时候这两个概念是一个意思,有时候却有很大区别。这里梳理下这两个术语所代表的具体含义。</p>
<h1 id="本文中数学符号及含义">本文中数学符号及含义</h1>
<table>
<thead>
<tr>
<th>符号</th>
<th>含义</th>
</tr>
</thead>
<tbody>
<tr>
<td>$O$</td>
<td>观测值</td>
</tr>
<tr>
<td>$\theta$</td>
<td>随机过程中的参数</td>
</tr>
<tr>
<td>$\hat{\theta}$</td>
<td>参数的估计</td>
</tr>
<tr>
<td>$P (O | \theta)$</td>
<td>概率</td>
</tr>
<tr>
<td>$L(\theta | O) $</td>
<td>释然</td>
</tr>
</tbody>
</table>
<h1 id="wiki中关于似然和概率的解释">wiki中关于“似然”和“概率”的解释</h1>
<ul>
<li>
<p>在频率推论中,似然函数(常常简称为似然)是一个在给定了数据以及模型中关于参数的函数。在非正式情况下,“似然”通常被用作“概率”的同义词。</p>
</li>
<li>
<p>在数理统计中,两个术语则有不同的意思。“概率”描述了给定模型参数后,描述结果的合理性,而不涉及任何观察到的数据。而“似然”则描述了给定了特定观测值后,描述模型参数是否合理。</p>
</li>
</ul>
<h1 id="例如">例如</h1>
<ul>
<li><strong>“概率”描述了给定模型参数后,描述结果的合理性,而不涉及任何观察到的数据。</strong></li>
</ul>
<blockquote>
<p>抛一枚均匀的硬币,拋20次,问15次拋得正面的可能性有多大?
这里的可能性就是”概率”,均匀的硬币就是给定参数$\theta=0.5$,“拋20次15次正面”是观测值$O$。求概率$P (H=15 | \theta=0.5) = ?$的概率。</p>
</blockquote>
<ul>
<li><strong>“似然”描述了给定了特定观测值后,描述模型参数是否合理。</strong></li>
</ul>
<blockquote>
<p>拋一枚硬币,拋20次,结果15次正面向上,问其为均匀的可能性?
这里的可能性就是”似然”,“拋20次15次正面”为观测值$O$为已知,参数$\theta=?$并不知道,求$L(\theta | H=15) = P (H=15 | \theta=0.5)$的最大化下的$\theta$ 值。</p>
</blockquote>
<h1 id="离散随机变量角度看待似然与概率">离散随机变量角度看待“似然”与“概率”</h1>
<p>当我们在处理离散型随机变量时候,(例如,掷10硬币的结果这样的数据时候)。这时候我们就可以根据观测到的结果计算这种结果出现的概率概率,当然这有一个前提是硬币是均匀的,和掷硬币的事件都是独立的。
这时我们想要计算的就是“概率”用$P(O | \theta)$来表示。换个角度可以理解为,当给定了特定的参数$\theta$时候,$P(O | \theta)$就是我们观测到$O$观测值时候的概率。
但是,当我们想来刻画一个实际的随机过程时候,我们常常并不知道$\theta$参数是什么。我们只有观测值$O$,基于这个观测值我们往往想得到一个关于$\theta$的估计值$\hat{\theta}$。当给定$\theta$ 时候我们可以得到观测值$O$是$P (O | \theta)$。当然反过来,对于估计过程是在选择一个$\hat{\theta}$最大值,这个值就等价于真实观测值$O$的概率。换而言之,是在寻找一个值$\hat{\theta}$的最大化使得</p>
\[L(\theta | O) = P (O | \theta)\]
<p>这个$L(\theta | O)$就叫做似然函数。
很明显这是一个在已知观测值$O$为条件关于未知参数$\theta$的似然函数。</p>
<h1 id="从连续型随机变量角度看待似然与概率">从连续型随机变量角度看待“似然”与“概率”</h1>
<p>对于连续型随机变量与离散随机变量有一个非常重要的区别,就是人们不会去关注给定$\theta$后观测值$O$得概率。
因为,连续型随机变量存在无限多的结果(无限可分),这些结果是无法被穷尽的。
我们给出某一个结果对应的概率是没有意义的(连续型随机变量产生的结果是无限的,
落在任何一个“可能的结果”上的概率几乎都为0,也就是$P(O | \theta) = 0 )$。
当然,可以变换一种方式既给出落在结果区间范围上的概率,而非给出单个结果的概率,来解决这个问题。
对于观测值$O$,可以用概率密度函数(PDF:probability density function)来表示为:$f(O|\theta)$。
因此,在连续的情况下,我们通过最大化以下函数来估计观察到的结果$O$:</p>
\[L(\theta | O) = f(O | \theta)\]
<p>在这种情况下,我们不能在技术上断言我们找到最大化观察$O$的概率的参数值,因为我们最大化的是与观察结果$O$相关的PDF。</p>
<h1 id="似然和概率是站在两个角度看待问题">“似然”和“概率”是站在两个角度看待问题</h1>
<p>对于这个函数:</p>
\[P (O | \theta)\]
<p>输入有两个:$O$表示某一个具体的数据;$\theta$表示模型的参数。</p>
<ul>
<li>
<p>如果$\theta$是已知确定的,$O$是变量,这个函数叫做概率函数(probability function),它描述对于不同的样本$O$,其出现概率是多少。</p>
</li>
<li>
<p>如果$O$是已知确定的,$\theta$是变量,这个函数叫做似然函数(likelihood function), 它描述对于不同的模型参数,出现x这个样本点的概率是多少。</p>
</li>
</ul>
<h1 id="参考">参考:</h1>
<ul>
<li>
<p><a href="https:\en.wikipedia.org\wiki\Likelihood_function">Wiki: Likelihood function</a></p>
</li>
<li>
<p><a href="https:\stats.stackexchange.com\questions\2641\what-is-the-difference-between-likelihood-and-probability">What is the difference between “likelihood” and “probability”?</a></p>
</li>
</ul>Yang Fang很多时候基础的概念没搞懂,或者知道了只知道照着例子套用,知其然而不知其所以然。对于知识的理解只能到达会运用的层面。而对于知识的创新运用是远远不足的。 似然和概率在统计学中是经常见到的两个术语,有时候这两个概念是一个意思,有时候却有很大区别。这里梳理下这两个术语所代表的具体含义。 本文中数学符号及含义Python并行计算2017-11-10T00:00:00+00:002017-11-10T00:00:00+00:00https://yangfangs.github.io//2017/11/10/python-multiprocessing.md<p> 当你在使用 Python 进行 for 循环时候,发现计算的效率及其慢的时候,完全可以考虑使用 Python 的并行操作,毕竟这么多核心不用也是一种浪费。还有 Python 标准库提供了并行库相当的简洁好用。</p>
<h1 id="python中的标准库">Python中的标准库</h1>
<p> Python 标准库中有两个提供并行计算一个是<a href="https://docs.python.org/3.6/library/multiprocessing.html">multiprocessing</a>, 另一个是 <a href="https://docs.python.org/3.6/library/threading.html">threading</a>,推荐使用 multiprocessing, 当只有I/O密集型操作时候才有用到threading的必要。当然由于 Python 的 <a href="https://docs.python.org/2.7/glossary.html#term-global-interpreter-lock">Global interpreter lock</a>存在,导致 Python 不能进行并行编程,只能进行并发编程。</p>
<h1 id="多进程multiprocessing的使用">多进程<code class="language-plaintext highlighter-rouge">multiprocessing</code>的使用</h1>
<ul>
<li>multiprocessing模块还是非常好用的。</li>
</ul>
<h2 id="单参数函数执行并计算">单参数函数执行并计算</h2>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">multiprocessing</span>
<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">x</span><span class="o">*</span><span class="n">x</span>
<span class="c1"># Get all cores
</span><span class="n">cores</span> <span class="o">=</span> <span class="n">multiprocessing</span><span class="p">.</span><span class="n">cpu_count</span><span class="p">()</span>
<span class="c1"># start a pool
</span><span class="n">pool</span> <span class="o">=</span> <span class="n">multiprocessing</span><span class="p">.</span><span class="n">Pool</span><span class="p">(</span><span class="n">processes</span><span class="o">=</span><span class="n">cores</span><span class="p">)</span>
<span class="n">tasks</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="mi">4</span><span class="p">,</span><span class="mi">5</span><span class="p">]</span>
<span class="c1"># do parallel calculate
</span><span class="k">print</span><span class="p">(</span><span class="n">pool</span><span class="p">.</span><span class="nb">map</span><span class="p">(</span><span class="n">f</span><span class="p">,</span><span class="n">tasks</span><span class="p">))</span>
</code></pre></div></div>
<blockquote>
<p>其实<code class="language-plaintext highlighter-rouge">map</code>函数在执行的时候就已经是并行操作了,只不过<code class="language-plaintext highlighter-rouge">multiprocessing</code>模块集成了<code class="language-plaintext highlighter-rouge">map</code>方法</p>
</blockquote>
<h2 id="多参数函数执行并行计算">多参数函数执行并行计算</h2>
<h3 id="python3-中使用starmap方法来实现">Python3 中使用<code class="language-plaintext highlighter-rouge">starmap</code>方法来实现</h3>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">multiprocessing</span>
<span class="k">def</span> <span class="nf">add</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">):</span>
<span class="k">return</span> <span class="n">x</span><span class="o">+</span><span class="n">y</span>
<span class="c1"># Get all worker processes
</span><span class="n">cores</span> <span class="o">=</span> <span class="n">multiprocessing</span><span class="p">.</span><span class="n">cpu_count</span><span class="p">()</span>
<span class="c1"># Start all worker processes
</span><span class="n">pool</span> <span class="o">=</span> <span class="n">multiprocessing</span><span class="p">.</span><span class="n">Pool</span><span class="p">(</span><span class="n">processes</span><span class="o">=</span><span class="n">cores</span><span class="p">)</span>
<span class="n">x1</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">5</span><span class="p">))</span>
<span class="n">y1</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">5</span><span class="p">))</span>
<span class="n">tasks</span> <span class="o">=</span> <span class="p">[(</span><span class="n">x</span><span class="p">,</span><span class="n">y</span><span class="p">)</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">x1</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">y1</span><span class="p">]</span>
<span class="k">print</span><span class="p">(</span><span class="n">pool</span><span class="p">.</span><span class="n">starmap</span><span class="p">(</span><span class="n">add</span><span class="p">,</span><span class="n">tasks</span><span class="p">))</span>
</code></pre></div></div>
<h3 id="python2-中需要一个函数对多参数函数包装下">Python2 中需要一个函数对多参数函数包装下</h3>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">multiprocessing</span>
<span class="k">def</span> <span class="nf">add</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">):</span>
<span class="k">return</span> <span class="n">x</span><span class="o">+</span><span class="n">y</span>
<span class="k">def</span> <span class="nf">merge_add</span><span class="p">(</span><span class="n">args</span><span class="p">):</span>
<span class="k">return</span> <span class="n">add</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span>
<span class="c1"># Get all worker processes
</span><span class="n">cores</span> <span class="o">=</span> <span class="n">multiprocessing</span><span class="p">.</span><span class="n">cpu_count</span><span class="p">()</span>
<span class="c1"># Start all worker processes
</span><span class="n">pool</span> <span class="o">=</span> <span class="n">multiprocessing</span><span class="p">.</span><span class="n">Pool</span><span class="p">(</span><span class="n">processes</span><span class="o">=</span><span class="n">cores</span><span class="p">)</span>
<span class="n">x1</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">5</span><span class="p">))</span>
<span class="n">y1</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">5</span><span class="p">))</span>
<span class="n">tasks</span> <span class="o">=</span> <span class="p">[(</span><span class="n">x</span><span class="p">,</span><span class="n">y</span><span class="p">)</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">x1</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">y1</span><span class="p">]</span>
<span class="k">print</span><span class="p">(</span><span class="n">pool</span><span class="p">.</span><span class="nb">map</span><span class="p">(</span><span class="n">merge_add</span><span class="p">,</span><span class="n">tasks</span><span class="p">))</span>
</code></pre></div></div>
<h1 id="参考">参考</h1>
<ul>
<li><a href="https://stackoverflow.com/questions/5442910/python-multiprocessing-pool-map-for-multiple-arguments">Python multiprocessing for mutiple arguments</a></li>
<li><a href="https://docs.python.org/2/library/multiprocessing.html">multiprocessing</a></li>
</ul>Yang Fang 当你在使用 Python 进行 for 循环时候,发现计算的效率及其慢的时候,完全可以考虑使用 Python 的并行操作,毕竟这么多核心不用也是一种浪费。还有 Python 标准库提供了并行库相当的简洁好用。CentOS 中安装 Python32017-10-01T00:00:00+00:002017-10-01T00:00:00+00:00https://yangfangs.github.io//2017/10/01/install-python3-in-centOS<p> 由于 CentOS 默认使用的是 Python2, 并且默认的 yum 官方源不提供 Python3 的镜像,所以只有自己动手安装了。</p>
<h1 id="安装必须的依赖">安装必须的依赖</h1>
<ul>
<li>首先在管理员权限下安装好所有的依赖,不然会出现各种各样的报错和问题。</li>
</ul>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>yum <span class="nt">-y</span> <span class="nb">install </span>zlib-devel bzip2-devel libffi-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gcc make
</code></pre></div></div>
<h1 id="获取-python3-的最新安装包">获取 Python3 的最新安装包</h1>
<ul>
<li>这里获取的是当前最新的稳定版本 <code class="language-plaintext highlighter-rouge">3.7.1</code> 可以在 <a href="https://www.python.org/">Python官网</a>获取最新发布版本。</li>
</ul>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
wget <span class="s2">"https://www.python.org/ftp/python/3.7.1/Python-3.7.1.tar.xz"</span>
</code></pre></div></div>
<h1 id="解压和安装">解压和安装</h1>
<ul>
<li>解压</li>
</ul>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
xz <span class="nt">-d</span> Python-3.7.1.tar.xz
<span class="nb">tar </span>xf Python-3.7.1.tar <span class="nt">-C</span> /usr/local/src/
<span class="nb">cd</span> /usr/local/src/Python-3.7.1/
</code></pre></div></div>
<ul>
<li>安装</li>
</ul>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./configure <span class="nt">--prefix</span><span class="o">=</span>/opt/python3
make all
make <span class="nb">install
</span>make clean
</code></pre></div></div>
<h1 id="查看安装好的-python3-版本信息">查看安装好的 Python3 版本信息</h1>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>/opt/python3/bin/python3 <span class="nt">-V</span>
</code></pre></div></div>
<h1 id="软链接到当先用户">软链接到当先用户</h1>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="nb">ln</span> <span class="nt">-s</span> /opt/python3/bin/python3 /usr/bin/python3
</code></pre></div></div>
<h1 id="脚本使用python3">脚本使用python3</h1>
<ul>
<li>在 Python 脚本前加入注释行</li>
</ul>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#! /usr/bin/env python3</span>
</code></pre></div></div>
<h1 id="参考">参考</h1>
<ul>
<li><a href="http://blog.csdn.net/hot_vc/article/details/48265835">CentOs6 安装PYTHON3</a></li>
<li><a href="http://blog.csdn.net/wang1144/article/details/52411373">centos 6.5安装python3.4</a></li>
</ul>Yang Fang 由于 CentOS 默认使用的是 Python2, 并且默认的 yum 官方源不提供 Python3 的镜像,所以只有自己动手安装了。Python 中 map(), filter(), reduce() 和 zip() 函数的用法2017-08-23T00:00:00+00:002017-08-23T00:00:00+00:00https://yangfangs.github.io//2017/08/23/python-map-zip-filter-reduce<p> Python 自带模块的数据结构屈指可数,list是一个随时都在用的数据结构,对list进行操作python内置了几个函数对python的list进行操作时候非常方便。</p>
<h1 id="map函数作用于list每一个元素">map()函数——作用于list每一个元素</h1>
<ul>
<li>
<p>map()是 Python 内置的高阶函数,它接收一个函数 f() 和一个 list,并通过把函数 f 依次作用在 list 的每个元素上,得到一个新的 list 并返回。(这个函数与 R 中的 lapply 非常相似)</p>
</li>
<li>
<p>用法map(function, sequence)</p>
</li>
</ul>
<h2 id="轻松转换-list-中元素类型">轻松转换 list 中元素类型:</h2>
<ul>
<li>例如 chr 类型转换成 int</li>
</ul>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="n">l</span> <span class="o">=</span> <span class="p">[</span><span class="s">'1'</span><span class="p">,</span><span class="s">'2'</span><span class="p">,</span><span class="s">'3'</span><span class="p">,</span><span class="s">'4'</span><span class="p">]</span>
<span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="nb">int</span><span class="p">,</span><span class="n">l</span><span class="p">))</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">2</span><span class="p">]:</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">]</span>
</code></pre></div></div>
<h2 id="编写独立函数作用与-list-中每一个元素">编写独立函数作用与 list 中每一个元素:</h2>
<ul>
<li>例如对 list 中每一个元素求平方</li>
</ul>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">x</span><span class="o">**</span><span class="mi">2</span>
<span class="n">l</span> <span class="o">=</span><span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="mi">4</span><span class="p">]</span>
<span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="n">f</span><span class="p">,</span><span class="n">l</span><span class="p">))</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">3</span><span class="p">]:</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">9</span><span class="p">,</span> <span class="mi">16</span><span class="p">]</span>
</code></pre></div></div>
<h2 id="使用匿名函数操作">使用匿名函数操作:</h2>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">l</span> <span class="o">=</span><span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="mi">4</span><span class="p">]</span>
<span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="o">**</span><span class="mi">2</span><span class="p">,</span> <span class="n">l</span><span class="p">))</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">4</span><span class="p">]:</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">9</span><span class="p">,</span> <span class="mi">16</span><span class="p">]</span>
</code></pre></div></div>
<h2 id="同时操作两个-list并行非多核运算">同时操作两个 list(并行非多核运算)</h2>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="n">l</span> <span class="o">=</span><span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="mi">4</span><span class="p">]</span>
<span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">,</span><span class="n">y</span><span class="p">:</span> <span class="n">x</span><span class="o">+</span><span class="n">y</span><span class="p">,</span><span class="n">l</span><span class="p">,</span><span class="n">l</span><span class="p">))</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">5</span><span class="p">]:</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">8</span><span class="p">]</span>
</code></pre></div></div>
<blockquote>
<p>注:python3 和 python2 中map()的返回值不一样, python2 中直接返回列表,python需要加list()转换取值。</p>
</blockquote>
<h1 id="filter函数筛选函数">filter()函数——筛选函数</h1>
<ul>
<li>按照 function 函数的规则在列表 sequence 中筛选数据</li>
<li>用法:filter(function, sequence)</li>
</ul>
<h2 id="筛选-list-中符合条件的值">筛选 list 中符合条件的值</h2>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">l</span> <span class="o">=</span><span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="mi">4</span><span class="p">]</span>
<span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="o">></span><span class="mi">2</span><span class="p">,</span> <span class="n">l</span><span class="p">)</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">6</span><span class="p">]:</span> <span class="p">[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">]</span>
</code></pre></div></div>
<h2 id="filter-与-map-返回值不同">filter() 与 map() 返回值不同</h2>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">l</span> <span class="o">=</span><span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="mi">4</span><span class="p">]</span>
<span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="o">></span><span class="mi">2</span><span class="p">,</span> <span class="n">l</span><span class="p">)</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">8</span><span class="p">]:</span> <span class="p">[</span><span class="bp">False</span><span class="p">,</span> <span class="bp">False</span><span class="p">,</span> <span class="bp">True</span><span class="p">,</span> <span class="bp">True</span><span class="p">]</span>
</code></pre></div></div>
<h1 id="reduce求积累运算">reduce()——求积累运算</h1>
<ul>
<li>
<p>reduce函数功能是将 sequence 中数据,按照 function 函数操作,如将列表第一个数与第二个数进行 function 操作,得到的结果和列表中下一个数据进行 function 操作,一直循环下去…</p>
</li>
<li>
<p>用法reduce(function, sequence):</p>
</li>
</ul>
<h2 id="求积累和">求积累和</h2>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">l</span> <span class="o">=</span><span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">]</span>
<span class="nb">reduce</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">,</span><span class="n">y</span><span class="p">:</span> <span class="n">x</span><span class="o">+</span><span class="n">y</span><span class="p">,</span> <span class="n">l</span><span class="p">)</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">10</span><span class="p">]:</span> <span class="mi">10</span>
</code></pre></div></div>
<h1 id="zip打包函数">zip()打包函数</h1>
<ul>
<li>
<p>zip()是 Python 的一个内建函数,它接受一系列可迭代的对象作为参数,将对象中对应的元素打包成一个个tuple(元组),然后返回由这些tuples组成的list(列表)。若传入参数的长度不等,则返回 list 的长度和参数中长度最短的对象相同。利用*号操作符,可以将list unzip(解压)。</p>
</li>
<li>
<p>用法: zip(list,list)</p>
</li>
</ul>
<h2 id="zip基本用法">zip()基本用法</h2>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">l1</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">]</span>
<span class="n">l2</span> <span class="o">=</span> <span class="p">[</span><span class="s">'a'</span><span class="p">,</span> <span class="s">'b'</span><span class="p">,</span> <span class="s">'c'</span><span class="p">,</span> <span class="s">'d'</span><span class="p">]</span>
<span class="nb">zip</span><span class="p">(</span><span class="n">l1</span><span class="p">,</span><span class="n">l2</span><span class="p">)</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">12</span><span class="p">]:</span> <span class="p">[(</span><span class="mi">1</span><span class="p">,</span> <span class="s">'a'</span><span class="p">),</span> <span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="s">'b'</span><span class="p">),</span> <span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="s">'c'</span><span class="p">),</span> <span class="p">(</span><span class="mi">4</span><span class="p">,</span> <span class="s">'d'</span><span class="p">)]</span>
</code></pre></div></div>
<h2 id="使用逆过程">使用<code class="language-plaintext highlighter-rouge">*</code>逆过程</h2>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">l1</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">]</span>
<span class="n">l2</span> <span class="o">=</span> <span class="p">[</span><span class="s">'a'</span><span class="p">,</span> <span class="s">'b'</span><span class="p">,</span> <span class="s">'c'</span><span class="p">,</span> <span class="s">'d'</span><span class="p">]</span>
<span class="n">zip_l1_l2</span> <span class="o">=</span> <span class="nb">zip</span><span class="p">(</span><span class="n">l1</span><span class="p">,</span><span class="n">l2</span><span class="p">)</span>
<span class="nb">zip</span><span class="p">(</span><span class="o">*</span><span class="n">zip_l1_l2</span><span class="p">)</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">17</span><span class="p">]:</span> <span class="p">[(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">),</span> <span class="p">(</span><span class="s">'a'</span><span class="p">,</span> <span class="s">'b'</span><span class="p">,</span> <span class="s">'c'</span><span class="p">,</span> <span class="s">'d'</span><span class="p">)]</span>
</code></pre></div></div>
<h2 id="zip-构造字典">zip 构造字典</h2>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">l1</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">]</span>
<span class="n">l2</span> <span class="o">=</span> <span class="p">[</span><span class="s">'a'</span><span class="p">,</span> <span class="s">'b'</span><span class="p">,</span> <span class="s">'c'</span><span class="p">,</span> <span class="s">'d'</span><span class="p">]</span>
<span class="n">zip_l1_l2</span> <span class="o">=</span> <span class="nb">zip</span><span class="p">(</span><span class="n">l1</span><span class="p">,</span><span class="n">l2</span><span class="p">)</span>
<span class="nb">dict</span><span class="p">(</span><span class="n">zip_l1_l2</span><span class="p">)</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">18</span><span class="p">]:</span> <span class="p">{</span><span class="mi">1</span><span class="p">:</span> <span class="s">'a'</span><span class="p">,</span> <span class="mi">2</span><span class="p">:</span> <span class="s">'b'</span><span class="p">,</span> <span class="mi">3</span><span class="p">:</span> <span class="s">'c'</span><span class="p">,</span> <span class="mi">4</span><span class="p">:</span> <span class="s">'d'</span><span class="p">}</span>
</code></pre></div></div>Yang Fang Python 自带模块的数据结构屈指可数,list是一个随时都在用的数据结构,对list进行操作python内置了几个函数对python的list进行操作时候非常方便。JavaFx 及其在 IntelliJ IDEA 中配置使用及发布2017-07-28T00:00:00+00:002017-07-28T00:00:00+00:00https://yangfangs.github.io//2017/07/28/JavaFx-SceneBuilder<p> 学习一下 GUI 的开发,想来想去几种语言中比较合适的还是 Java。有几个优点是其他几种语言无法比拟的。</p>
<ol>
<li>
<p>Java语言的跨平台性,Linux,Windows,Mac OS,只要装个 Jre,都能运行。</p>
</li>
<li>
<p>面向对象编程和很多非常优秀的接口可以使用。</p>
</li>
<li>
<p>性能优秀并不用像使用C语言一样担心内存问题。</p>
</li>
<li>
<p>与 C++ 相比,的确 C++ 实现的GUI更加绚丽和元素更加丰富,但是对于科学计算用的 GUI 设计, Java 足够了。</p>
</li>
</ol>
<p> 所以学习一下 Java 的 GUI 开发,发现 Java 的 GUI 开发框架也是不少的,主要流的 GUI 开发框架有两个:</p>
<ul>
<li>
<p><a href="https://docs.oracle.com/javase/tutorial/uiswing/">Swing</a> 同时有支持可视化开发的 <a href="https://eclipse.org/windowbuilder/">WindowBuilder</a></p>
</li>
<li>
<p><a href="http://docs.oracle.com/javase/8/javase-clienttechnologies.htm">JavaFX</a> 同时有支持可视化开发的 <a href="http://www.oracle.com/technetwork/java/javase/downloads/sb2download-2177776.html">SceneBuilder</a></p>
</li>
</ul>
<p> JavaFX 是 SUN 公司在 2007 年 JavaOne 大会上首次对外公布,在 2001 年发布了 2.0 版本。而 AWT 和 Swing API 早在 1996(JDK 1.0) 年就形成,在 1998 年JDK 1.2全面添加。JavaFx已经被集成到 Java 8 的标准库中。所以还是非常值得探究学习的。
本文记录下 JavaFX SceneBuilder 安装及其在 IntelliJ IDEA 中的配置</p>
<h2 id="需要安装">需要安装</h2>
<ul>
<li>
<p><a href="http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html">JDK8(Java SE Development Kit 8)</a></p>
</li>
<li>
<p><a href="http://www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-2133155.html">JRE8(Java SE Runtime Environment 8)</a></p>
</li>
<li>
<p><a href="https://www.jetbrains.com/idea/download/#section=windows">IntelliJ IDEA (Community)</a></p>
</li>
<li>
<p><a href="http://www.oracle.com/technetwork/java/javase/downloads/sb2download-2177776.html">JavaFX Scene Builder</a></p>
</li>
</ul>
<h2 id="intellij-idea-中配置-scenebuilder">IntelliJ IDEA 中配置 SceneBuilder</h2>
<h3 id="设置-scenebuilder-路径">设置 SceneBuilder 路径</h3>
<ul>
<li>依次打开<code class="language-plaintext highlighter-rouge">File->Setting->Languages&Frameworks->JavaFx</code>设置启动 SceneBuilder 的路径</li>
</ul>
<p><img src="/images/posts/java/javafx_install_fig1.png" alt="fig1" /></p>
<h3 id="创建一个-javafx-项目">创建一个 JavaFx 项目</h3>
<ul>
<li>依次打开 <code class="language-plaintext highlighter-rouge">File->New->project</code>选择JavaFx,如果安装多个版本 SDK, 选择 Java8。</li>
</ul>
<p><img src="/images/posts/java/javafx_install_fig2.png" alt="fig2" /></p>
<h3 id="在-intellij-idea-中打开-scenebuilder">在 IntelliJ IDEA 中打开 SceneBuilder</h3>
<ul>
<li>创建完成后会自动生成一个样本项目。包括两个类一个逻辑控制类 <code class="language-plaintext highlighter-rouge">Controller</code> 和一个主函数类 <code class="language-plaintext highlighter-rouge">Main</code> 以及一个 <code class="language-plaintext highlighter-rouge">fxml</code> 格式的配置文件,右键点击 <code class="language-plaintext highlighter-rouge">fxml</code> 格式文件选择 <code class="language-plaintext highlighter-rouge">Open in SceneBuilder</code>:</li>
</ul>
<p><img src="/images/posts/java/javafx_install_fig2.png" alt="fig3" /></p>
<h3 id="发布-javafx-应用">发布 JavaFx 应用</h3>
<ul>
<li>依次打开<code class="language-plaintext highlighter-rouge">File->Project Structure->Artifacts->"+"->JavaFx Application->From module...</code></li>
</ul>
<p><img src="/images/posts/java/javafx_install_fig3.png" alt="fig4" /></p>
<ul>
<li>选择输出地址以及JavaFx选项下面的一些说明都得填写。</li>
</ul>
<p><img src="/images/posts/java/javafx_install_fig4.png" alt="fig5" /></p>Yang Fang 学习一下 GUI 的开发,想来想去几种语言中比较合适的还是 Java。有几个优点是其他几种语言无法比拟的。概率导论学习记录(一)2017-04-10T00:00:00+00:002017-04-10T00:00:00+00:00https://yangfangs.github.io//2017/04/10/probability-chapter-one<p> 近来发现师兄书架里面有一本非常好的概率论方面的书——图灵社区图书 <a href="https://book.douban.com/subject/26694188/">概率导论</a>,翻一下觉得讲的很不错,
于是问师兄借过来拜读下,读完第一章感觉写的浅显易懂。不像国内的太多概率论书,上来就是一大堆的定理、推论、证明、过程和公式,显得过于枯燥,概率导论作为入门来说是一本非常好的书。
这里记录下学习概率导论每一章中非常基础但是又非常重要的知识点,方便以后快速查询。</p>
<h1 id="第一章样本空间与概率">第一章样本空间与概率</h1>
<h2 id="11集合和集合运算">1.1集合和集合运算</h2>
<ul>
<li>将研究的对象放在一起就形成了<strong>集合</strong>,而这些对象就称为集合的<strong>元素</strong>。</li>
</ul>
<h3 id="111集合中常用的符合及意义">1.1.1集合中常用的符合及意义</h3>
<ul>
<li>集合中常用的符合以及意义如下:</li>
</ul>
<table>
<thead>
<tr>
<th>符号</th>
<th>意义</th>
</tr>
</thead>
<tbody>
<tr>
<td>$S$</td>
<td>表示一个集合</td>
</tr>
<tr>
<td>$\Omega$</td>
<td>表示空间</td>
</tr>
<tr>
<td>$S^c$</td>
<td>表示集合的补集</td>
</tr>
<tr>
<td>$x \in S$</td>
<td>表示元素$x$属于集合$s$</td>
</tr>
<tr>
<td>$x \not\in S$</td>
<td>表示元素$x$不属于集合$s$</td>
</tr>
<tr>
<td>$S \subset T$</td>
<td>表示集合$S$包含于集合$T$</td>
</tr>
<tr>
<td>$S \not\subset T$</td>
<td>表示集合$S$不包含集合$T$</td>
</tr>
<tr>
<td>$S \cup T$</td>
<td>表示集合$S$和集合$T$的并集</td>
</tr>
<tr>
<td>$S \cap T$</td>
<td>表示集合$S$和集合$T$的交集</td>
</tr>
</tbody>
</table>
<h3 id="112集合代数运算">1.1.2集合代数运算</h3>
<ul>
<li>集合常用的代数运算:</li>
</ul>
<table>
<thead>
<tr>
<th>基本代数运算</th>
</tr>
</thead>
<tbody>
<tr>
<td>$S \cup T = T \cup S$</td>
</tr>
<tr>
<td>$S \cup (T \cup U) = (S \cup U) \cup T$</td>
</tr>
<tr>
<td>$S \cap (T \cup U) = (S \cap T) \cup (S \cap U)$</td>
</tr>
<tr>
<td>$S \cup (T \cap U) = (S \cup T) \cap (S \cup U)$</td>
</tr>
<tr>
<td>$(S^c)^c = S$</td>
</tr>
<tr>
<td>$S \cap S^c = \emptyset$</td>
</tr>
<tr>
<td>$S \cup \Omega = \Omega$</td>
</tr>
<tr>
<td>$ S \cap \Omega = S$</td>
</tr>
</tbody>
</table>
<h2 id="12-概率模型">1.2 概率模型</h2>
<ul>
<li>概率模型是对不确定的现象的数学描述。每一个概率模型都关联着一个<strong>试验</strong>,这个试验将产生一个试验<strong>结果</strong>。该试验的所有可能结果形成一个<strong>样本空间$\Omega$</strong>。某些试验结果的集合,即样本空间的子集称为<strong>事件</strong>。</li>
</ul>
<h3 id="121概率模型的基本构成">1.2.1概率模型的基本构成</h3>
<blockquote>
<p><strong>样本空间</strong>,常用$\Omega$表示,是一个试验所有的可能得结果的集合。</p>
<p><strong>概率律</strong>,为试验结果的集合$A$(称为<strong>事件</strong>)确定一个非负数$P(A)$(称为事件$A$的<strong>概率</strong>)。而这个非负数刻画了我们对事件$A$的认识或所产生的信念的程度。</p>
</blockquote>
<h3 id="122概率律">1.2.2概率律</h3>
<ul>
<li>
<p>当给定一个事件$A$同时确定一个数$P(A)$称为事件$A$的<strong>概率</strong>并且满以下几条公理:</p>
</li>
<li>
<p><strong>概率公理</strong></p>
<blockquote>
<p><strong>非负性</strong>:对一切事物$A$,满足$P(A) \geq 0$。</p>
<p><strong>可加性</strong>:设$A$和$B$为两个互不相交的集合(即互不相容事件),则他们的并集满足:</p>
<p>$P(A \cup B)=P(A)+P(B)$</p>
<p>若$A_1$,$A_2$,…是互不相容的事件序列,则他们的并集满足:</p>
<p>$P(A_1 \cup A_2 \cup …)=P(A_1)+P(A_2)+…$.</p>
<p><strong>归一化</strong>:整个样本空间$\Omega$(为必然事件)的概率为$1$,即$P(\Omega)=1$</p>
</blockquote>
</li>
</ul>
<h3 id="123离散模型">1.2.3离散模型</h3>
<ul>
<li>利用概率律的可加性公理可以得到离散概率律:</li>
<li><strong>离散概率律</strong>
<blockquote>
<p>设空间样本是由有限个可能的结果组成,则事件的概率可由组成这个事件的试验结果的概率律所决定,事件$ \left\{ s_1,s_2,…,s_n \right\} $的概率是$P(s_i)$之和,即:
$P(\left\{s_1,s_2,…,s_n\right\})= P(s_1)+P(s_2)+…+P(s_n)$.</p>
</blockquote>
</li>
<li><strong>离散均匀概率律(古典概型)</strong>
<blockquote>
<p>设样本空间由$n$个等可能性的试验结果组成,因此每个试验结果组成的事件(基本事件)的概率是相等的。由此得到:</p>
<p>\(P(A)=\frac{含事件A的试验结果}{n}\).</p>
</blockquote>
</li>
</ul>
<h3 id="124-概率律的性质">1.2.4 概率律的性质</h3>
<ul>
<li>考虑一个概率律,令$A$,$B$,$C$为事件,由概率公理可以推倒出以下性质:
<blockquote>
<p>(a)若$A \subset B$则 $P(A) \leq P(B)$.</p>
<p>(b)$P(A \cup B)=P(A)+P(B) - P(A \cap B)$.</p>
<p>(c)$P(A \cup B) \geq P(A) + P(B)$.</p>
<p>(d)$P(A \cup B \cup C) = P(A) + P(A^c \cap B) + P(A^C \cap B^c \cap C)$.</p>
</blockquote>
</li>
</ul>
<h2 id="13条件概率">1.3条件概率</h2>
<ul>
<li>条件概率是在给定的<code class="language-plaintext highlighter-rouge">部分信息</code>的基础上对试验结果的一种推断,条件概率的性质有如下三条。
<blockquote>
<ol>
<li>设事件$B$满足$P(B) > 0$,则给定$B$之下,事件$A$的条件概率由下式给出:
\(P (A | B) = \frac {P(A \cap B)}{P(B)}\).</li>
</ol>
<p>这个条件概率律在同一个样本空间$\Omega$上给出了一个新的(条件)概率律。凡是现有的概率律的所有性质对这个条件概率都是适用的。</p>
<ol>
<li>
<p>由于条件概率所关心的事件都是事件$B$的子事件,可以把条件概率看成$B$上的概率律,即把时间$B$看成全空间或必然事件</p>
</li>
<li>
<p>当试验$\Omega$是有限集,并且所有的试验结果为等可能的情况下,条件概率律可由下式给出:</p>
</li>
</ol>
\[P(A|B)= \frac{事件A \cap B的试验结果数}{事件B的试验结果数}\]
</blockquote>
</li>
<li>条件概率的乘法规则:
<blockquote>
<p>假定所有涉及的条件概率都是正的,我们有:
\(P(\bigcap_{i=1}^nA_i) = P(A_1)P(A_2|A_1)P(A_3|A_1 \cap A_2)...P(A_n|\bigcap_{i=1}^{n-1}A_i)\).</p>
</blockquote>
</li>
</ul>
<h2 id="14-全概率定理和贝叶斯准则">1.4 全概率定理和贝叶斯准则</h2>
<h3 id="141全概率定理">1.4.1全概率定理</h3>
<ul>
<li>全概率定理如下:
<blockquote>
<p>设$A_1$,$A_2$,$A_n$是一组互不相容的事件,形成样本空间的一个分割(每一个试验结果必定使得其中一个事件发生)。又假定对每一个事件$i$,$P(A_i)>0$。则对于任意事件$B$下列公式成立</p>
\[P(B)=P(A_1 \cap B)+...+P(A_n \cap B)\\\ =P(A_1)P(B|A_1)+...+P(A_n)P(B|A_n)\]
</blockquote>
</li>
</ul>
<h3 id="142贝叶斯准则">1.4.2贝叶斯准则</h3>
<ul>
<li>全概率定理是与贝叶斯准则联系在一起的,
贝叶斯准则将形如 $P(A|B)$ 的条件概率与形如
$P(B|A)$ 的条件概率联系起来。
贝叶斯准则如下:
<blockquote>
<p>设$A_1$,$A_2$,$A_n$是一组互不相容的事件,形成样本空间的一个分割(每一个试验结果必定使得其中一个事件发生)。又假定对每一个事件$i$,$P(A_i)>0$。则对于任意事件$B$,只要它满足$P(B)>0$下列公式成立
\(P(A_i|B)= \frac{P(A_i)P(B|A_i)}{P(B)}= \frac{P(A_i)P(B|A_i)}{P(A_1)P(B|A_1)+...+P(A_n)P(B|A_n)}\)</p>
</blockquote>
</li>
</ul>
<h2 id="15独立性">1.5独立性</h2>
<h3 id="151独立性的主要结论">1.5.1独立性的主要结论</h3>
<ul>
<li>若两个事件$A$和$B$相互独立,可以有以下结论:
<blockquote>
<p>1.若两个事件$A$和$B$相互独立,如果他们满足:</p>
<p>$P(A \cap B)=P(A)P(B)$.</p>
<p>若$B$还满足$P(B)>0$,则独立性等价于</p>
\[P(A | B)=P(A)\]
<p>2.若$A$与$B$相互独立,则$A$与$B^c$也相互独立。</p>
<p>3.设事件$C$满足$P(C)>0$,两个事件$A$和$B$称为给定$C$条件下的条件独立,如果他们满足:</p>
\[P(A \cap B|C)=P(A|C)P(B|C)\]
<p>若进一步假设$P(B \cap C)>0$,则$A$和$B$在给定$C$的条件下的条件独立性与下面的条件是等价的</p>
\[P(A|B \cap C)=P(A|C)\]
<p>独立性并不蕴含条件独立,反之亦然。</p>
</blockquote>
</li>
</ul>
<h3 id="152一组事件的独立性">1.5.2一组事件的独立性</h3>
<ul>
<li>两个事件相互独立性的概念可以推广到多个事件的相互独立性,并可以得到多事件相互独立的定义:
<blockquote>
<p>设$A_1…A_n$为$n$个事件。若他们满足</p>
<p>$P\big( \displaystyle\bigcap_{i \in S}A_i\big)=\prod_{i \in S}P(A_i)$对于${1,2,3,…,n }$的任意子集$S$成立,</p>
<p>则称$A_1,…,A_n$为相互独立的事件。</p>
</blockquote>
</li>
</ul>
<h2 id="16计数准则">1.6计数准则</h2>
<h3 id="161计数准则">1.6.1计数准则</h3>
<ul>
<li>计数准则基于分段计数的原则,这是计数的最基本的方法。计数准则如下:
<blockquote>
<p>考虑由$r$个阶段组成的一个试验。假设:</p>
<p>(a) 在第一个阶段有$n_1$个可能的结果;</p>
<p>(b) 对于第一个阶段任何一个结果,在第二个阶段有$n_2$个可能的结果;</p>
<p>(c) 一般地,在前$r-1$个阶段的任何一个结果,在接下来第$r$个阶段有$n_r$个结果,则在$r$个阶段的试验中一共有:</p>
\[n_1n_2n_3 \cdots n_r\]
<p>个试验结果</p>
</blockquote>
</li>
</ul>
<h3 id="162计数法">1.6.2计数法</h3>
<ul>
<li>常用的计数法有,排列,组合,以及分割数,有如下汇总:
<blockquote>
<ol>
<li>$n$个对象的<strong>排列数</strong>:$n!$.</li>
<li>$n$个对象取$k$个对象的<strong>排列数</strong>:$\frac{n!}{(n-k)!}$.</li>
<li>$n$个对象取$k$个对象的<strong>组合数</strong>:$\big(^n _k\big)= \frac{n!}{k!(n-k)!}$.</li>
<li>$n$个对象分割成$r$个组的<strong>分割数</strong>,其中$i$个组具有$n_i$个对象:
$\big(^n _{n_1,n_2,…,n_r}= \frac{n!}{n_1!n_2! \cdots n_r!} \big)$.</li>
</ol>
</blockquote>
</li>
</ul>
<h1 id="参考">参考:</h1>
<ul>
<li><a href="https://book.douban.com/subject/26694188/">概率导论</a></li>
</ul>Yang Fang 近来发现师兄书架里面有一本非常好的概率论方面的书——图灵社区图书 概率导论,翻一下觉得讲的很不错, 于是问师兄借过来拜读下,读完第一章感觉写的浅显易懂。不像国内的太多概率论书,上来就是一大堆的定理、推论、证明、过程和公式,显得过于枯燥,概率导论作为入门来说是一本非常好的书。 这里记录下学习概率导论每一章中非常基础但是又非常重要的知识点,方便以后快速查询。