{"id":259,"date":"2015-08-09T22:14:31","date_gmt":"2015-08-09T20:14:31","guid":{"rendered":"http:\/\/www.mlguru.cz\/?p=259"},"modified":"2017-08-13T10:32:49","modified_gmt":"2017-08-13T08:32:49","slug":"basnik","status":"publish","type":"post","link":"https:\/\/www.mlguru.com\/cs\/basnik\/","title":{"rendered":"St\u0159edov\u011bk um\u011bl\u00e9 inteligence skon\u010dil, seznamte se s neuronov\u00fdmi s\u00edt\u011bmi, kter\u00e9 um\u00ed ps\u00e1t b\u00e1sn\u011b"},"content":{"rendered":"

Nikdy jsem si nemyslel, \u017ee by jednou po\u010d\u00edta\u010de mohly nahradit v \u010dlov\u011bka v tak kreativn\u00edch a neexaktn\u00edch \u010dinnostech jako je malba nebo skl\u00e1d\u00e1n\u00ed b\u00e1sn\u00ed. Pomalu ale za\u010d\u00edn\u00e1m m\u011bnit n\u00e1zor.<\/p>\n

Nen\u00ed tomu dlouho, co se i v neodborn\u00fdch \u010dasopisech za\u010dalo ps\u00e1t o neuronov\u00fdch s\u00edt\u00edch, kter\u00e9 dok\u00e1\u017e\u00ed generovat obrazy<\/a>, p\u0159ipom\u00ednaj\u00edc\u00ed psychedelick\u00e9 v\u00fdtvory surrealistick\u00fdch mal\u00ed\u0159\u016f. Jak je to ale s automatick\u00fdm generov\u00e1n\u00edm text\u016f? N\u011bkolik \u010dl\u00e1nk\u016f na toto t\u00e9ma vzniklo (nap\u0159. http:\/\/karpathy.github.io\/2015\/05\/21\/rnn-effectiveness\/<\/a>), ale \u017e\u00e1dn\u00fd zat\u00edm nez\u00edskal takov\u00fd ohlas jako gener\u00e1tory obraz\u016f. Poj\u010fme tedy tuto oblast podpo\u0159it a uka\u017eme si, jak pomoc\u00ed um\u011bl\u00fdch neuronov\u00fdch s\u00edt\u00ed vytvo\u0159it automatick\u00e9ho b\u00e1sn\u00edka.<\/p>\n

Rekurentn\u00ed neuronov\u00e9 s\u00edt\u011b<\/h3>\n

V\u011bt\u0161ina neuronov\u00fdch s\u00edt\u00ed, kter\u00e9 jsou pou\u017e\u00edv\u00e1ny k anal\u00fdze i ke generov\u00e1n\u00ed obrazu, je dop\u0159edn\u00fdch<\/a>. To znamen\u00e1, \u017ee sign\u00e1l v neuronov\u00e9 s\u00edti m\u016f\u017ee proudit pouze jedn\u00edm sm\u011brem a nem\u016f\u017ee se vracet nebo se nap\u0159. to\u010dit ve smy\u010dce. S\u00edt\u011b, kter\u00e9 toto umo\u017e\u0148uj\u00ed se naz\u00fdvaj\u00ed rekurentn\u00ed<\/a>. Takov\u00e9 s\u00edt\u011b mnohem bl\u00ed\u017ee p\u0159ipom\u00ednaj\u00ed lidsk\u00fd mozek a p\u0159edev\u0161\u00edm maj\u00ed jednu z\u00e1sadn\u00ed vlastnost \u2014 a tou je mo\u017enost uchov\u00e1n\u00ed informace. Jin\u00fdmi slovy, rekurentn\u00ed s\u00edt\u011b maj\u00ed pam\u011b\u0165. D\u00edky tomu je vyjad\u0159ovac\u00ed schopnost t\u011bchto s\u00edt\u00ed v\u011bt\u0161\u00ed ne\u017e u dop\u0159edn\u00fdch neuronov\u00fdch s\u00edt\u00ed, a je dokonce dok\u00e1z\u00e1no<\/a>, \u017ee je ekvivalentn\u00ed Turingovu stroji.<\/p>\n

Pam\u011b\u0165 nen\u00ed nutn\u00e1 v\u017edy, pokud v\u0161ak chceme pracovat se sekven\u010dn\u00edmi daty jako jsou texty, bez pam\u011bti se obejdeme jen st\u011b\u017e\u00ed. Nev\u00fdhodou rekurentn\u00edch neuronov\u00fdch s\u00edt\u00ed je v\u0161ak obt\u00ed\u017enost tr\u00e9nov\u00e1n\u00ed. Zat\u00edm existuje jen n\u011bkolik m\u00e1lo architektur rekurentn\u00edch s\u00edt\u00ed, kter\u00e9 dok\u00e1\u017eeme tr\u00e9novat efektivn\u011b. Jednou z nich je takzvan\u00e1 Long Short-Term Memory<\/a> (LSTM), kterou pou\u017eijeme jako stavebn\u00ed prvek na\u0161eho automatick\u00e9ho b\u00e1sn\u00edka.<\/p>\n

Jedna pam\u011b\u0165ov\u00e1 bu\u0148ka Long Short-Term Memory je zn\u00e1zorn\u011bna na obr\u00e1zku 1, p\u0159evzat\u00e9m z http:\/\/www.willamette.edu\/~gorr\/classes\/cs449\/lstm.html<\/a>.<\/p>\n

\"Long<\/a>

Obr\u00e1zek 1: Long Short-Term Memory.<\/p><\/div>\n

Jej\u00ed j\u00e1dro tvo\u0159\u00ed stavov\u00fd neuron spolu se zp\u011btnovazebn\u00ed smy\u010dkou, ozna\u010den\u00fd jako \u201ecell state\u201c. D\u00edky zp\u011btn\u00e9 vazb\u011b je neuron schopn\u00fd uchov\u00e1vat informaci, kter\u00e1 byla p\u0159ivedena na vstup i pot\u00e9, co je vstup odpojen. P\u0159ipojen\u00ed a odpojen\u00ed vstupu je ovl\u00e1d\u00e1no neuronem ozna\u010den\u00fdm jako \u201einput gate\u201c. Podobn\u011b je aktivace v\u00fdstupu \u0159\u00edzena neuronem ozna\u010den\u00fdm jako \u201eoutput gate\u201c.<\/p>\n

Automatick\u00fd b\u00e1sn\u00edk<\/h3>\n

Abychom mohli natr\u00e9novat neuronovou s\u00ed\u0165 generuj\u00edc\u00ed b\u00e1sn\u011b, pot\u0159ebujeme j\u00ed uk\u00e1zat, jak takov\u00e9 b\u00e1sn\u011b maj\u00ed vypadat. K tomu by ide\u00e1ln\u011b poslou\u017eil velk\u00fd korpus b\u00e1sn\u00ed. Vzhledem k tomu, \u017ee jsem \u017e\u00e1dn\u00fd velk\u00fd nem\u011bl k dispozici, pou\u017eil jsem datab\u00e1zi p\u00eds\u0148ov\u00fdch text\u016f ze serveru supermusic.sk<\/a>. Sice nejde o b\u00e1sn\u011b, ale p\u00eds\u0148ov\u00e9 texy maj\u00ed k poezii bl\u00edzko. Pomoc\u00ed jednoduch\u00e9 detekce jazyka byly odstran\u011bny ne\u010desk\u00e9 texy a vznikl tak 10 MB velk\u00fd textov\u00fd korpus. Kv\u016fli odd\u011blen\u00ed jednotliv\u00fdch p\u00edsn\u00ed byly jejich n\u00e1zvy ve form\u011b nadpis\u016f p\u0159evedeny na kapit\u00e1lky a samotn\u00e9 texty na mal\u00e1 p\u00edsmena.<\/p>\n

Neuronov\u00e1 s\u00ed\u0165 pro generov\u00e1n\u00ed b\u00e1sn\u00ed je sestavena z p\u011bti vrstev. Vstupn\u00ed vrstva sest\u00e1v\u00e1 ze 157 neuron\u016f (co\u017e je po\u010det r\u016fzn\u00fdch znak\u016f v korpusu). N\u00e1sleduj\u00ed t\u0159i vz\u00e1jemn\u011b \u00fapln\u011b propojen\u00e9 vrstvy LSTM, p\u0159i\u010dem\u017e ka\u017edou vrstvu tvo\u0159\u00ed 1024 bun\u011bk. V\u00fdstup je tvo\u0159en 157neuronovou softmax<\/a> vrstvou, jej\u00ed\u017e v\u00fdstupem je pravd\u011bpodobnostn\u00ed rozd\u011blen\u00ed p\u0159es 157 stav\u016f. Architektura cel\u00e9 s\u00edt\u011b je zn\u00e1zorn\u011bna na obr\u00e1zku 2.<\/p>\n

\"Automatick\u00fd<\/a>

Obr\u00e1zek 2: Automatick\u00fd b\u00e1sn\u00edk.<\/p><\/div>\n

S\u00ed\u0165 je tr\u00e9nov\u00e1na tak, aby na z\u00e1klad\u011b historie a p\u00edsmene na aktu\u00e1ln\u00ed pozici v korpusu predikovala n\u00e1sleduj\u00edc\u00ed p\u00edsmeno. D\u00edky tomu, \u017ee s\u00ed\u0165 pracuje na \u00farovni p\u00edsmen a ne slov, jak by se nab\u00edzelo, je mo\u017en\u00e9 se nau\u010dit i form\u00e1tov\u00e1n\u00ed, specifick\u00e9 pro z\u00e1pis p\u00eds\u0148ov\u00fdch text\u016f.<\/p>\n

Vzr\u016fstaj\u00edc\u00ed popularita neuronov\u00fdch s\u00edt\u00ed s sebou p\u0159inesla celou \u0159adu hotov\u00fdch n\u00e1stroj\u016f a knihoven. Nen\u00ed tak u\u017e na\u0161t\u011bst\u00ed pot\u0159eba programovat v\u0161e od p\u00edky. Pro natr\u00e9nov\u00e1n\u00ed v\u00fd\u0161e popsan\u00e9 s\u00edt\u011b byla pou\u017eita implementace Char-rnn<\/a> vyu\u017e\u00edvaj\u00edc\u00ed framework Torch.<\/a><\/p>\n

V\u00fdsledek nen\u00ed v\u016fbec \u0161patn\u00fd<\/h3>\n

Pot\u00e9, co byla s\u00ed\u0165 natr\u00e9novan\u00e1 na p\u00eds\u0148ov\u00e9m korpusu, bylo mo\u017en\u00e9 ji pou\u017e\u00edt pro generov\u00e1n\u00ed nov\u00fdch text\u016f. Pro \u00fa\u010dely testov\u00e1n\u00ed byla inicializov\u00e1na n\u00e1hodn\u011b, ale je mo\u017en\u00e9 ji inicializovat nap\u0159\u00edklad i vlo\u017een\u00edm n\u00e1zvu b\u00e1sn\u011b nebo po\u010d\u00e1te\u010dn\u00edch ver\u0161\u016f. Teoreticky m\u016f\u017ee natr\u00e9novan\u00e1 s\u00edt vygenerovat neomezen\u00e9 mno\u017estv\u00ed r\u016fzn\u00fdch text\u016f, nechal jsem tedy vygenerovat n\u011bkolik des\u00edtek b\u00e1sn\u00ed a vybral p\u00e1r nejlep\u0161\u00edch. B\u00e1sn\u011b jsou p\u0159esn\u011b v t\u00e9 podob\u011b, v jak\u00e9 je vygenerovala s\u00ed\u0165.<\/p>\n

Zde je jedna z nejlep\u0161\u00edch:<\/p>\n

P\u00cdSE\u0147 PODZIMU<\/p>\n

pro\u010d se nedoraz\u00ed\u0161?
\ntelefon\u00e1t nen\u00ed nad\u011bje
\npo\u0159\u00e1d doma na t\u00fdhle planet\u011b
\n\u010das pro tebe st\u00e1le dupe svou<\/p>\n

na na na...<\/p>\n

j\u00e1 v\u00edm, \u017ee v duchu prozrazuju sn\u00e1\u0159
\npozn\u00e1m \u017ee nejsem nic v\u00edc<\/p><\/blockquote>\n

Soud\u011b podle tohoto p\u0159\u00edkladu dok\u00e1zala s\u00ed\u0165 bezchybn\u011b pou\u017e\u00edt \u010deskou slovn\u00ed z\u00e1sobu, morfologii i syntax. Nejzaj\u00edmav\u011bj\u0161\u00ed ale je, \u017ee i obsahov\u011b d\u00e1v\u00e1 b\u00e1se\u0148 v\u00edcem\u00e9n\u011b smysl a nen\u00ed \u00fapln\u011b hloup\u00e1. Dalo by se nam\u00edtnou, \u017ee mohlo doj\u00edt k p\u0159etr\u00e9nov\u00e1n\u00ed a s\u00ed\u0165 jen opakuje ver\u0161e, kter\u00e9 vid\u011bla p\u0159i tr\u00e9nov\u00e1n\u00ed. Pokud se v\u0161ak pod\u00edv\u00e1me do korpusu, \u017e\u00e1dn\u00fd z pou\u017eit\u00fdch ver\u0161\u016f se tam nevyskytuje.<\/p>\n

Je z\u0159ejm\u00e9, \u017ee je tato neuronov\u00e1 s\u00ed\u0165 schopn\u00e1 pochopit i netrivi\u00e1ln\u00ed s\u00e9mantick\u00e9 konstrukce. P\u0159\u00edkladem m\u016f\u017ee b\u00fdt p\u0159irovn\u00e1n\u00ed \u201evz\u00e1cn\u00fd jako list\u00ed\u201c v n\u00e1sleduj\u00edc\u00ed b\u00e1sni. Op\u011bt se jedn\u00e1 o souslov\u00ed, kter\u00e9 se v korpusu v\u016fbec nevyskytlo:<\/p>\n

V\u0160E PR\u00dd JE M\u016eJ H\u0158\u00cdCH<\/p>\n

j\u00e1 nechci sly\u0161et cht\u00ed\u010d
\ntak vz\u00e1cn\u00fd jako list\u00ed
\nslzy v o\u010d\u00edch m\u00e1m
\na m\u011bl bych \u0159v\u00e1t<\/p>\n

kdo mi to v\u00e1\u0161n\u011b ukr\u00fdv\u00e1
\nnetu\u0161\u00edm, ka\u017ed\u00fd je tu s\u00e1m
\njen p\u00e1r let jsem p\u0159\u00edsahal
\n\u017ee pro mne m\u00e1 smysl \u017e\u00edt.<\/p><\/blockquote>\n

Obecn\u011b se d\u00e1 \u0159\u00edci, \u017ee \u010d\u00edm krat\u0161\u00ed b\u00e1se\u0148 je, t\u00edm v\u011bt\u0161\u00ed smysl d\u00e1v\u00e1. Je to samoz\u0159ejm\u011b d\u00e1no t\u00edm, \u017ee v del\u0161\u00edch textech je v\u011bt\u0161\u00ed prostor pro odbo\u010den\u00ed od t\u00e9matu. Ob\u010das m\u00e1 s\u00ed\u0165 tendenci vytvo\u0159it velmi kr\u00e1tkou b\u00e1se\u0148. Pokud to ud\u011bl\u00e1, t\u00e9m\u011b\u0159 v\u017edy se jedn\u00e1 o n\u011bco poveden\u00e9ho a \u00fasm\u011bvn\u00e9ho. Zde jsou dva p\u0159\u00edklady:<\/p>\n

NEJSI TAMARA<\/p>\n

jsi ludmila<\/p><\/blockquote>\n

NEJHEZ\u010c\u00cd D\u00c1REK<\/p>\n

50 k\u010ds, tak m\u00e1m t\u011b r\u00e1d<\/p><\/blockquote>\n

N\u011bkdy se stane, \u017ee si s\u00ed\u0165 vymysl\u00ed nov\u00e9 slovo. Pokud k tomu dojde, \u010dasto se v\u0161ak jedn\u00e1 o velmi zaj\u00edmav\u00fd novotvar jako v n\u00e1zvu n\u00e1sleduj\u00edc\u00ed b\u00e1sn\u011b:<\/p>\n

D\u017d\u00cdNOV\u00c1 POKROPAN\u00cd<\/p>\n

tak p\u0159esta\u0148, smi\u0159 se s t\u00edm
\nmoje \u0159e\u010d, m\u016fj dech, b\u00fdt sv\u00e1 bez pen\u011bz
\nmysl\u00edm, \u017ee jsi \u017eena co dech ti d\u00e1
\ns tebou u\u017e nev\u00edm co s t\u00edm<\/p>\n

kdy\u017e ve\u010der temn\u011b z\u00e1\u0159\u00ed
\nchu\u0165 ve mn\u011b z\u016fstane
\njen v klidu mi tv\u00fd kroky jdou
\nu\u017e d\u00e1vno se sm\u011bje<\/p>\n

zkou\u0161\u00edm se vr\u00e1tit
\nnechci se pt\u00e1t
\nhled\u00e1m skr\u00fd\u0161
\nka\u017edou noc<\/p>\n

a pak se ztrat\u00ed
\nneum\u00edm lh\u00e1t
\na nejde to vr\u00e1tit<\/p><\/blockquote>\n

A na z\u00e1v\u011br n\u011bkolik dal\u0161\u00edch v\u00fdtvor\u016f r\u016fzn\u00fdch kvalit:<\/p>\n

NEJSI<\/p>\n

sn\u00edh a mraziv\u00e1
\nv\u00edm v\u00edm<\/p>\n

pro\u010d zrovna jedna z tebe chce \u017e\u00edt
\nto nen\u00ed mo\u017en\u00fd ty to v\u00ed\u0161
\na j\u00e1 m\u00e1m ho r\u00e1d
\njen tak promlouvej<\/p>\n

ztr\u00e1c\u00edm s\u00edlu \u017e\u00edt<\/p><\/blockquote>\n

PRO\u010c SE MI ZD\u00c1<\/p>\n

j\u00e1 v\u00edm, \u017ee jsem s\u00e1m
\na tak to m\u00e1m r\u00e1d
\na tak se mi zd\u00e1
\n\u017ee t\u011b nechci b\u00edt<\/p>\n

j\u00e1 v\u00edm, \u017ee t\u011b nechci b\u00edt<\/p>\n

j\u00e1 v\u00edm, \u017ee jsem jen tv\u016fj<\/p><\/blockquote>\n

Z\u00c1TON\u00cd<\/p>\n

nikdo nem\u00e1 koloto\u010d.
\nkr\u00e1sn\u00e9 pa\u017ee host\u00ed,
\ntroub\u00ed \u017elutejch kon\u00ed
\na k ve\u010deru opil\u00ed.<\/p>\n

d\u00e9\u0161\u0165 o\u010dima \u017eivot hled\u00e1,
\nmus\u00edm odeslat je d\u00e1l,
\nproto\u017ee jsi lacin\u00e1
\np\u0159ece dob\u0159e u\u017e nen\u00ed.<\/p>\n

\u010dlov\u011bk n\u00e1\u0161 kruh co pust\u00fd sen
\nchce\u0161-li vedle nosit chlast.
\noh, jak miliady t\u0159e\u0161t\u00ed?
\npak mi \u0159ekli: nebude\u0161, tak se st\u0159e\u017e<\/p><\/blockquote>\n

KR\u00c1LOVNA V\u010cEL<\/p>\n

nebude to divn\u011b zn\u00edt,
\npozor na tv\u016fj strach
\nnesm\u011bj se hned
\na hned od lid\u00ed<\/p>\n

pro\u010d pocit mi ned\u00e1\u0161,
\nnechci to hned
\nchce\u0161 mi \u0159\u00edkat zat\u00edm star\u00fd posl\u00e1n\u00ed
\nkdy\u017e to ale nev\u00ed\u0161, nevad\u00ed<\/p>\n

posledn\u00ed r\u00e1j,
\nhled\u00e1m nahoru a dol\u016f
\n\u017ee je nev\u011brn\u00e1
\na\u0165 nen\u00ed d\u00e1l<\/p>\n

venku je den
\ntak pro\u010d se boj\u00edme
\nsp\u00e1t<\/p>\n

dob\u0159e v\u00edme
\n\u017ee je n\u011bco v n\u00ed
\na j\u00e1 u\u017e to v\u00edm<\/p>\n

ooou j\u00e9<\/p><\/blockquote>\n

Pokud v\u00e1s zaj\u00edmaj\u00ed dal\u0161\u00ed p\u0159\u00edklady po\u010d\u00edta\u010dem generovan\u00fdch b\u00e1sn\u00ed, m\u016f\u017eete si zdarma st\u00e1hnout mou b\u00e1snickou sb\u00edrku Poezie um\u011bl\u00e9ho sv\u011bta<\/a>. Pro vygenerov\u00e1n\u00ed obsa\u017een\u00fdch b\u00e1sn\u00ed byl pou\u017eit\u00fd vylep\u0161en\u00fd algortitmus napsan\u00fd v TensorFlow<\/a> a jako tr\u00e9novac\u00ed sada poslou\u017eily b\u00e1sn\u011b ze serveru P\u00edsm\u00e1k.cz<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

Nikdy jsem si nemyslel, \u017ee by jednou po\u010d\u00edta\u010de mohly nahradit v \u010dlov\u011bka v tak kreativn\u00edch a neexaktn\u00edch \u010dinnostech jako je malba nebo skl\u00e1d\u00e1n\u00ed b\u00e1sn\u00ed. Pomalu ale za\u010d\u00edn\u00e1m m\u011bnit n\u00e1zor. Nen\u00ed tomu dlouho, co se i v neodborn\u00fdch \u010dasopisech za\u010dalo ps\u00e1t o neuronov\u00fdch s\u00edt\u00edch, kter\u00e9 dok\u00e1\u017e\u00ed generovat obrazy, p\u0159ipom\u00ednaj\u00edc\u00ed psychedelick\u00e9 v\u00fdtvory surrealistick\u00fdch mal\u00ed\u0159\u016f. Jak je to […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[10],"tags":[],"class_list":["post-259","post","type-post","status-publish","format-standard","hentry","category-nlp"],"_links":{"self":[{"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/posts\/259","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/comments?post=259"}],"version-history":[{"count":24,"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/posts\/259\/revisions"}],"predecessor-version":[{"id":441,"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/posts\/259\/revisions\/441"}],"wp:attachment":[{"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/media?parent=259"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/categories?post=259"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/tags?post=259"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}