{"id":328,"date":"2017-08-13T14:38:45","date_gmt":"2017-08-13T12:38:45","guid":{"rendered":"http:\/\/www.mlguru.cz\/?p=328"},"modified":"2017-08-13T14:38:45","modified_gmt":"2017-08-13T12:38:45","slug":"why-and-when-the-analogies-in-the-word2vec-algorithms-work","status":"publish","type":"post","link":"https:\/\/www.mlguru.com\/cs\/why-and-when-the-analogies-in-the-word2vec-algorithms-work\/","title":{"rendered":"Pro\u010d a kdy funguj\u00ed slovn\u00ed analogie v algoritmech word2vec"},"content":{"rendered":"

Od publikov\u00e1n\u00ed prvn\u00edho algoritmu Tom\u00e1\u0161e Mikolova, p\u0159ev\u00e1d\u011bj\u00edc\u00edho slova na vektory se zaj\u00edmav\u00fdmi vlastnostmi (psal jsem o n\u011bm zde<\/a>), ub\u011bhlo ji\u017e n\u011bkolik let. St\u00e1le se v\u0161ak lid\u00e9 sna\u017e\u00ed v\u00edce \u010di m\u00e9n\u011b uspokojiv\u011b vysv\u011btlit, jak je mo\u017en\u00e9, \u017ee \"kings<\/strong> \u2013 king<\/strong> + queen<\/strong>\u00a0 = queens<\/strong>\". <\/em>P\u0159ed \u010dasem napsal podobn\u00fd text na sv\u00e9m blogu<\/a>\u00a0Piotr Migdal. \u010cl\u00e1nek je inspirativn\u00ed, ale obsahuje \u0159adu nepodlo\u017een\u00fdch tvrzen\u00ed a p\u0159edpoklad\u016f. R\u00e1d bych zde prezentoval alternativn\u00ed zd\u016fvodn\u011bn\u00ed vych\u00e1zej\u00edc\u00ed ze stejn\u00fdch my\u0161lenek, kter\u00e9 v\u0161ak bude precizn\u011bj\u0161\u00ed.<\/p>\n

Jedn\u00edm z\u00a0hlavn\u00edm p\u0159\u00ednos\u016f v\u011bt\u0161iny takzvan\u00fdch word embedding algoritm\u016f (cbow, skipgram, glove, atd.) je schopnost projektovat lexik\u00e1ln\u00ed jednotky (nej\u010dast\u011bji slova) do vektorov\u00e9ho prostoru, ve kter\u00e9m morfologick\u00e9, syntaktick\u00e9 i n\u011bkter\u00e9 s\u00e9mantick\u00e9 vlastnosti t\u011bchto slov zachov\u00e1vaj\u00ed line\u00e1rn\u00ed z\u00e1vislosti. Znamen\u00e1 to, \u017ee kdy\u017e nap\u0159\u00edklad vektor slova king<\/em><\/strong> ode\u010dteme od vektoru slova kings<\/strong><\/em>, dostaneme vektor, kter\u00fd lze v\u00fdznamov\u011b ch\u00e1pat jako vektor p\u0159echodu od singul\u00e1ru k\u00a0plur\u00e1lu. Pokud tento vektor p\u0159i\u010dteme k\u00a0libovoln\u00e9mu substantivu v\u00a0singul\u00e1ru, dostaneme se v\u00a0dan\u00e9m vektorov\u00e9m prostoru velmi bl\u00edzko k\u00a0vektoru stejn\u00e9ho slova v\u00a0plur\u00e1lu. Tuto vlastnost ilustruje zn\u00e1m\u00fd obr\u00e1zek p\u0159evzat\u00fd z\u00a0p\u016fvodn\u00edho \u010dl\u00e1nku T. Mikolova.<\/p>\n

\"\"<\/a>

Obr\u00e1zek 1: Vektorov\u00e1 reprezentace slov. Zdroj: T. Mikolovov et al. :\u00a0Linguistic Regularities in Continuous Space Word Representations, NAACL 2013.<\/p><\/div>\n

Tohoto fenom\u00e9nu lze vyu\u017e\u00edt k\u00a0hled\u00e1n\u00ed slovn\u00edch analogi\u00ed. Mohli bychom se pt\u00e1t na slovo W<\/em><\/strong>, pro kter\u00e9 plat\u00ed: king<\/em><\/strong> se m\u00e1 ke kings<\/em><\/strong> jako queen<\/em><\/strong> k\u00a0W<\/em><\/strong>. V\u00a0obecnosti bychom cht\u011bli uk\u00e1zat, \u017ee pokud se slovo a<\/em><\/strong> m\u00e1 ke slovu A<\/em><\/strong> stejn\u011b jako slovo b<\/em><\/strong> ke slovu B<\/em><\/strong>, potom p\u0159ibli\u017en\u011b plat\u00ed va<\/sub><\/strong> \u2013 vA<\/sub><\/strong> = vb<\/sub><\/strong> \u2013 vB<\/sub><\/strong>, kde vx<\/sub><\/strong> je vektor reprezentuj\u00edc\u00ed slovo x<\/em><\/strong>, <\/em>z\u00edskan\u00fd n\u011bkter\u00fdm z\u00a0algoritm\u016f word2vec (viz Obr. 1).<\/p>\n

Word2vec je zobec\u0148uj\u00edc\u00ed pojmenov\u00e1n\u00ed pro metody p\u0159evodu slova na jeho vektorovou reprezentaci. Pro \u00fa\u010dely tohoto textu se zam\u011b\u0159me konkr\u00e9tn\u011b na nejzn\u00e1m\u011bj\u0161\u00ed algoritmus Skip-gram s\u00a0negativn\u00edm samplov\u00e1n\u00edm<\/a>.<\/p>\n

Abychom mohli dan\u00e9 tvrzen\u00ed dok\u00e1zat, je t\u0159eba se nejd\u0159\u00edve vypo\u0159\u00e1dat s v\u00e1gn\u00ed definic\u00ed vztahu \u201cm\u00edt se stejn\u011b k\u201d. To v\u011bt\u0161ina literatury zanedb\u00e1v\u00e1 a spokojuje se jen s intuic\u00ed. Pro v\u011bt\u0161\u00ed p\u0159ehlednost budeme v\u00a0n\u00e1sleduj\u00edc\u00edm textu tu\u010dnou kurz\u00edvou<\/em><\/strong> ozna\u010dovat lexik\u00e1ln\u00ed jednotky, kurz\u00edvou<\/em> jejich vlastnosti a tu\u010dn\u011b<\/strong> z\u00e1pis lexik\u00e1ln\u00ed jednotky v korpusu.<\/p>\n

Vyjd\u011bme z\u00a0Firthova<\/a> pohledu na lexik\u00e1ln\u00ed v\u00fdznam. Jeho zn\u00e1m\u00fd cit\u00e1t \u201eYou shall know a word by the company it keeps\u201c lze, voln\u011b p\u0159elo\u017eeno, ch\u00e1pat tak, \u017ee v\u00fdznam slova je d\u00e1n slovy, kter\u00e1 se s\u00a0n\u00edm \u010dasto poj\u00ed. Podle jeho tvrzen\u00ed tedy plat\u00ed, \u017ee dv\u011b slova maj\u00ed podobn\u00fd v\u00fdznam tehdy, kdy\u017e se v\u00a0textech \u010dasto vyskytuj\u00ed ve stejn\u00fdch kontextech. Jedn\u00e1 se o v\u00a0praxi \u010dasto vyu\u017e\u00edvan\u00fd pohled na lexik\u00e1ln\u00ed s\u00e9mantiku.<\/p>\n

V\u00fdznam slova a<\/em><\/strong> lze ve Firthov\u011b pojet\u00ed formalizovat jako soubor pravd\u011bpodobnost\u00ed SEM(a<\/em><\/strong>) = \u2329P(w <\/strong>| a<\/strong>): w<\/strong>\u00a0\u2208\u00a0L\u232a<\/strong>, kter\u00e9 vyjad\u0159uj\u00ed pravd\u011bpodobnost v\u00fdskytu slova w<\/strong> bl\u00edzko slova a<\/strong> v\u00a0jazyce L<\/strong>. Ten nech\u0165 je reprezentovan\u00fd dostate\u010dn\u00e9 velk\u00fdm jazykov\u00fdm korpusem. Bl\u00edzkost je ur\u010dena n\u011bjakou maxim\u00e1ln\u00ed vzd\u00e1lenost\u00ed v\u00a0textu (m\u011b\u0159eno po\u010dtem slov, kter\u00e1 je odd\u011bluj\u00ed).<\/p>\n

Dal\u0161\u00edm d\u016fle\u017eit\u00fdm poznatkem je Fregeho<\/a> princip kompozicionality<\/a>, kter\u00fd n\u00e1m \u0159\u00edk\u00e1, \u017ee v\u00fdznam slo\u017eit\u011bj\u0161\u00edho lexik\u00e1ln\u00edho v\u00fdrazu je d\u00e1n v\u00fdznamem d\u00edl\u010d\u00edch jednotek a zp\u016fsobem jejich kombinace. Tento princip se d\u00e1 vzt\u00e1hnout i na v\u00fdznam slov \u2013 slovo je nositelem souboru d\u00edl\u010d\u00edch v\u00fdznam\u016f, kter\u00e1 dohromady d\u00e1vaj\u00ed v\u00fdznam cel\u00e9ho slova. Nap\u0159\u00edklad slovo queen<\/em><\/strong> lze zjednodu\u0161en\u011b ch\u00e1pat jako anglick\u00e9 substantivum ozna\u010duj\u00edc\u00ed osobu, kter\u00e1 m\u00e1 sou\u010dasn\u011b vlastnosti b\u00fdt panovn\u00edkem s\u00a0kr\u00e1lovsk\u00fdm titulem<\/em> a b\u00fdt \u017eensk\u00e9ho pohlav\u00ed<\/em>.<\/p>\n

Vych\u00e1zej\u00edce z\u00a0p\u0159edchoz\u00edch dvou formulac\u00ed m\u016f\u017eeme tedy v\u00fdznam slova queen<\/em><\/strong> definovat pomoc\u00ed pravd\u011bpodobnost\u00ed\u2329P(<\/em>w<\/strong>i<\/sub> | je panovn\u00edk <\/em>\u2227\u00a0je \u017eensk\u00e9ho pohlav\u00ed<\/em> )\u232a<\/strong>, v\u00a0obecnosti jako\u00a0\u2329P(w<\/strong>i<\/sub> | a1<\/sub>, a2<\/sub>, \u2026 an<\/sub><\/em>)\u232a<\/strong>.<\/p>\n

Nyn\u00ed ji\u017e lze zadefinovat slovn\u00ed analogie: a<\/em><\/strong> se m\u00e1 k\u00a0A<\/em><\/strong> stejn\u011b jako b<\/em><\/strong> k\u00a0B<\/em><\/strong> tehdy, kdy\u017e existuj\u00ed vlastnosti a1<\/sub><\/em>... an<\/sub>, b1<\/sub><\/em>... bn<\/sub>, c<\/em> a d<\/em> takov\u00e9, \u017ee<\/p>\n

SEM(a<\/em><\/strong>) =\u2329P(w<\/strong>i<\/sub> | a1<\/sub>, a2<\/sub>, \u2026 an<\/sub>, c<\/em>)\u232a<\/strong><\/p>\n

SEM(A<\/em><\/strong>) =\u2329P(w<\/strong>i<\/sub> | a1<\/sub>, a2<\/sub>, \u2026 an<\/sub>, d<\/em>)\u232a<\/strong><\/p>\n

SEM(b<\/em><\/strong>) =\u2329P(w<\/strong>i<\/sub> | b1<\/sub>, b2<\/sub>, \u2026 bm<\/sub>, c<\/em>)\u232a<\/strong><\/p>\n

SEM(B<\/em><\/strong>) =\u2329P(w<\/strong>i<\/sub> | b1<\/sub>, b2<\/sub>, \u2026 bm<\/sub>, d<\/em>)\u232a<\/strong>.<\/p>\n

Pokud budou platit n\u00e1sleduj\u00edc\u00ed statistick\u00e9 nez\u00e1vislosti v\u00fdskyt\u016f vlastnost\u00ed slov v\u00a0korpusu<\/p>\n

P(a1<\/sub>, a2<\/sub>, \u2026 an<\/sub><\/em>) \u22a5 P(c)<\/p>\n

P(a1<\/sub>, a2<\/sub>, \u2026 an<\/sub><\/em>) \u22a5 P(d)<\/p>\n

P(b1<\/sub>, b2<\/sub>, \u2026 bm<\/sub><\/em>) \u22a5 P(c)<\/p>\n

P(b1<\/sub>, b2<\/sub>, \u2026 bm<\/sub><\/em>) \u22a5 P(d)<\/p>\n

Dostaneme pro v\u0161echna wi<\/sub><\/strong><\/p>\n

\\dfrac{P(\\mathbf{w_i} | \\mathbf{a})}{P(\\mathbf{w_i} | \\mathbf{A})} = \\dfrac{P(\\mathbf{w_i} | \\mathbf{b})}{P(\\mathbf{w_i} | \\mathbf{B})} <\/p>\n

nebo\u0165<\/p>\n

\\dfrac{P(\\mathbf{w_i} | \\mathbf{a})}{P(\\mathbf{w_i} | \\mathbf{A})} = \\dfrac{P(\\mathbf{w_i} | a_1 \\dots a_n, c)}{P(\\mathbf{w_i} | a_1 \\dots a_n, d)} = \\dfrac{P(\\mathbf{w_i}, a_1 \\dots a_n, c)P(a_1 \\dots a_n, d)}{P(\\mathbf{w_i}, a_1 \\dots a_n, d)P(a_1 \\dots a_n, c)} = <\/p>\n

=\\dfrac{P(a_1 \\dots a_n, c | \\mathbf{w_i})P(\\mathbf{w_i})P(a_1 \\dots a_n)P(d)}{P(a_1 \\dots a_n, d | \\mathbf{w_i})P(\\mathbf{w_i})P(a_1 \\dots a_n)P(c)} = \\dfrac{P(a_1 \\dots a_n | \\mathbf{w_i})P( c | \\mathbf{w_i})P(d)}{P(a_1 \\dots a_n | \\mathbf{w_i})P( d | \\mathbf{w_i})P(c)} = <\/p>\n

= \\dfrac{P( c | \\mathbf{w_i})P(d)}{P( d | \\mathbf{w_i})P(c)} = \\dfrac{P(\\mathbf{w_i}|P(b_1 \\dots b_m, c)}{P(\\mathbf{w_i}|P(b_1 \\dots b_m, d)} = \\dfrac{P(\\mathbf{w_i} | \\mathbf{b})}{P(\\mathbf{w_i} | \\mathbf{B})}.<\/p>\n

Nyn\u00ed ji\u017e lze analogie snadno zd\u016fvodnit<\/p>\n

\\dfrac{P(\\mathbf{w_i} | \\mathbf{a})}{P(\\mathbf{w_i} | \\mathbf{A})} = \\dfrac{P(\\mathbf{w_i} | \\mathbf{b})}{P(\\mathbf{w_i} | \\mathbf{B})}<\/p>\n

\\mbox{log} \\Bigg( \\dfrac{P(\\mathbf{w_i} | \\mathbf{a})}{P(\\mathbf{w_i})} \\cdot \\dfrac{P(\\mathbf{w_i})}{P(\\mathbf{w_i} | \\mathbf{A})} \\Bigg) = \\mbox{log} \\Bigg( \\dfrac{P(\\mathbf{w_i} | \\mathbf{b})}{P(\\mathbf{w_i})} \\cdot \\dfrac{P(\\mathbf{w_i})}{P(\\mathbf{w_i} | \\mathbf{B})} \\Bigg)<\/p>\n

\\mbox{log} \\dfrac{P(\\mathbf{w_i} | \\mathbf{a})}{P(\\mathbf{w_i})}-\\mbox{log} \\dfrac{P(\\mathbf{w_i} | \\mathbf{A})}{P(\\mathbf{w_i})}=\\mbox{log} \\dfrac{P(\\mathbf{w_i} | \\mathbf{b})}{P(\\mathbf{w_i})}-\\mbox{log} \\dfrac{P(\\mathbf{w_i} | \\mathbf{B})}{P(\\mathbf{w_i})}<\/p>\n

P\u0159i\u010dem\u017e\u00a0z definice pointwise mutual information<\/a>\u00a0plat\u00ed<\/p>\n

\\mbox{PMI}(\\mathbf{w_i},\\mathbf{a}) -\\mbox{PMI}(\\mathbf{w_i},\\mathbf{A}) =\\mbox{PMI}(\\mathbf{w_i},\\mathbf{b}) -\\mbox{PMI}(\\mathbf{w_i},\\mathbf{B}).<\/p>\n

Te\u010f vyu\u017eijeme v\u00fdsledek pr\u00e1ce\u00a0Levy \u00a0& Yoav Goldberg, 2014<\/a>, kde bylo uk\u00e1z\u00e1no, \u017ee algoritmus Skip-gram s negativn\u00edm samplov\u00e1n\u00edm\u00a0aproximuje rozklad matice slov a kontext\u016f, kde jednotliv\u00e9 bu\u0148ky matice odpov\u00eddaj\u00ed (a\u017e na konstatn\u00ed posuv) pointwise mutual information. Auto\u0159i Arora & kol., 2015<\/a>\u00a0sice ukazuj\u00ed, \u017ee aproximace se bl\u00ed\u017e\u00ed skute\u010dnosti jen u vektor\u016f vysok\u00e9 dimenze a navrhuj\u00ed vlastn\u00ed d\u016fkaz, pro pot\u0159eby tohoto \u010dl\u00e1nku n\u00e1m v\u0161ak tvrzen\u00ed sta\u010d\u00ed:<\/p>\n

\\mbox{PMI}(\\mathbf{w},\\mathbf{a}) \\approx v_\\mathbf{w} \\cdot v_\\mathbf{a}.<\/p>\n

Odsud ji\u017e dostaneme k\u00fd\u017eenou line\u00e1rn\u00ed z\u00e1vislost vektor\u016f, odpov\u00eddaj\u00edc\u00ed slovn\u00edm analogi\u00edm:<\/p>\n

v_\\mathbf{a} - v_\\mathbf{A} = v_\\mathbf{b} - v_\\mathbf{B}.<\/p>\n

Probl\u00e9mem d\u016fkazu je p\u0159edpoklad nez\u00e1vislosti v\u00fdskytu jev\u016f v korpusu, kter\u00fd samoz\u0159ejm\u011b ve v\u011bt\u0161in\u011b p\u0159\u00edpad\u016f neplat\u00ed. To je n\u011bco, \u010d\u00edm se \u017e\u00e1dn\u00fd ze mn\u011b zn\u00e1m\u00fdch d\u016fkaz\u016f analogi\u00ed v algoritmech word2vec kv\u016fli v\u00e1gn\u00ed nebo chyb\u011bj\u00edc\u00ed definici v\u00fdznamu slov nezab\u00fdval. Je mo\u017en\u00e9, \u017ee existuje siln\u011bj\u0161\u00ed d\u016fkaz, kter\u00fd p\u0159edpoklad nez\u00e1vislosti jev\u016f nepot\u0159ebuje. Druhou mo\u017enost\u00ed je, \u017ee slovn\u00ed analogie funguj\u00ed p\u0159esn\u011b jen v p\u0159\u00edpad\u011b statisticky nez\u00e1visl\u00fdch jev\u016f. Ani pro jedno v\u0161ak nem\u00e1m zd\u016fvodn\u011bn\u00ed, a proto nech\u00e1v\u00e1m ot\u00e1zku otev\u0159enou pro p\u0159\u00edpadn\u00e9 z\u00e1jemce z \u0159ad \u010dten\u00e1\u0159\u016f.<\/p>\n

Pro tuto chv\u00edli v\u0161ak nechme stranou exaktn\u00ed d\u016fkazy a pod\u00edvejme na n\u011bkolik p\u0159\u00edklad\u016f analogi\u00ed napo\u010d\u00edtan\u00fdch algoritmem skip-gram na korpusu \u010desk\u00e9ho internetu.<\/p>\n\n\n\n\n\t\n\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t
+ slovo<\/th>- slovo<\/th>+ slovo<\/th>nejbli\u017e\u0161\u00ed slovo v\u00fdsledku<\/th>kosinov\u00e1 podobnost<\/th>\n<\/tr>\n<\/thead>\n
kr\u00e1lovna<\/td>kr\u00e1l<\/td>\u0159idi\u010d<\/td>\u0159idi\u010dka<\/td>0.785<\/td>\n<\/tr>\n
\u0158idi\u010dka<\/td>0.682<\/td>\n<\/tr>\n
cyklistka<\/td>0.625<\/td>\n<\/tr>\n
ryb\u00e1\u0159ka<\/td>ryb\u00e1\u0159<\/td>o\u0161et\u0159ovatel<\/td>o\u0161et\u0159ovatelka<\/td>0.692<\/td>\n<\/tr>\n
O\u0161et\u0159ovatelka<\/td>0.619<\/td>\n<\/tr>\n
canisterapeutka<\/td>0.605<\/td>\n<\/tr>\n
kr\u00e1lovna<\/td>kozel<\/td>\u0159idi\u010d<\/td>\u0159idi\u010dka<\/td>0.609<\/td>\n<\/tr>\n
\u010dty\u0159iadvacetilet\u00e1<\/td>0.532<\/td>\n<\/tr>\n
dvaadvacetilet\u00e1<\/td>0.511<\/td>\n<\/tr>\n
ryb\u00e1\u0159ka<\/td>kozel<\/td>o\u0161et\u0159ovatel<\/td>krotitelka<\/td>0.574<\/td>\n<\/tr>\n
o\u0161et\u0159ovatelka<\/td>0.547<\/td>\n<\/tr>\n
pasa\u010dka<\/td>0.541<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n\n

V\u00a0tabulce jsou v\u017edy uvedena t\u0159i slova, kter\u00e1 slou\u017e\u00ed jako dotaz a ke ka\u017ed\u00e9mu dotazu 3 nejbli\u017e\u0161\u00ed slova nalezen\u00e1 v\u00a0prostoru spolu s\u00a0kosinovou podobnost\u00ed. U prvn\u00edho p\u0159\u00edkladu, kde je dotazem v(kr\u00e1lovna<\/strong>) \u2013 v(kr\u00e1l<\/strong>) + v(\u0159idi\u010d<\/strong>) vid\u00edme, \u017ee analogie kr\u00e1sn\u011b funguj\u00ed a dv\u011b nejbli\u017e\u0161\u00ed nalezen\u00e1 slova jsou \u0159idi\u010dka<\/strong> a \u0158idi\u010dka<\/strong>. Ve druh\u00e9m p\u0159\u00edkladu byla z\u00e1m\u011brn\u011b pou\u017eita slova, pro kter\u00e1 p\u0159edpoklad nez\u00e1vislosti v\u00fdrazn\u011bji neplat\u00ed. Vlastnost b\u00fdt ryb\u00e1\u0159em<\/em> a b\u00fdt mu\u017e<\/em>, spolu jist\u011b pozitivn\u011b koreluj\u00ed. Podobn\u011b jako vlastnosti b\u00fdt o\u0161et\u0159ovatel<\/em> a b\u00fdt \u017eena<\/em>. P\u0159esto\u017ee kosinov\u00e1 podobnost nejbli\u017e\u0161\u00edho kandid\u00e1ta je ni\u017e\u0161\u00ed ne\u017e v\u00a0p\u0159edchoz\u00edm p\u0159\u00edpad\u011b, st\u00e1le je v\u00fdsledkem spr\u00e1vn\u00e1 odpov\u011b\u010f o\u0161et\u0159ovatelka<\/em><\/strong>. T\u0159et\u00ed p\u0159\u00edklad je extr\u00e9mn\u011bj\u0161\u00ed. Dotazem je zde v(kr\u00e1lovna<\/strong>) \u2013 v(kozel<\/strong>) + v(\u0159idi\u010d<\/strong>) a v\u00fdsledkem st\u00e1le \u0159idi\u010dka<\/strong>, i kdy\u017e u\u017e s\u00a0v\u00fdrazn\u011b ni\u017e\u0161\u00ed kosinovou podobnost\u00ed. Tento v\u00fdsledek je trochu zar\u00e1\u017eej\u00edc\u00ed a neodpov\u00edd\u00e1 na\u0161\u00ed intuici. Zd\u00e1 se, \u017ee vlastnost b\u00fdt mu\u017eem<\/em> a b\u00fdt \u017eenou<\/em> je natolik dominantn\u00ed, \u017ee p\u0159ev\u00e1\u017e\u00ed v\u0161echny ostatn\u00ed. Teprve posledn\u00ed p\u0159\u00edklad, kter\u00fd kombinuje extr\u00e9my p\u0159edchoz\u00edch uk\u00e1zek, dopadne jinak. Na dotaz v(ryb\u00e1\u0159ka<\/strong>) \u2013 v(kozel<\/strong>) + v(o\u0161et\u0159ovatel<\/strong>) najdeme jako nejbli\u017e\u0161\u00ed slovo krotitelka<\/strong>.<\/p>\n

T\u011bchto p\u00e1r p\u0159\u00edklad\u016f samoz\u0159ejm\u011b v\u016fbec nic nedokazuje. Je z\u00a0nich v\u0161ak vid\u011bt, \u017ee hled\u00e1n\u00ed analogi\u00ed pomoc\u00ed word2vec p\u0159\u00edstup\u016f je velmi odoln\u00e9 v\u016f\u010di \u0161umu. M\u016f\u017ee tedy dob\u0159e fungovat i v p\u0159\u00edpad\u011b, \u017ee by byl pro p\u0159esn\u00fd d\u016fkaz po\u017eadavek nez\u00e1vislosti jev\u016f nutn\u00fd. Nav\u00edc nep\u0159esnost\u00ed, kter\u00e9 do procesu hled\u00e1n\u00ed analogi\u00ed vstupuj\u00ed, je mnoho (velikost a zam\u011b\u0159en\u00ed korpusu, velikost reprezentuj\u00edc\u00edch vektor\u016f, apod.) a je jen ot\u00e1zkou, jak velkou chybu jednotliv\u011b zp\u016fsobuj\u00ed. Velk\u00e1 odolnost v\u016f\u010di nim je podle m\u00e9ho n\u00e1zoru zaji\u0161t\u011bna velkou \u0159\u00eddkost\u00ed vektorov\u00e9ho prostoru, ve kter\u00e9m analogie hled\u00e1me. Pro dotaz v(kr\u00e1lovna<\/strong>) \u2013 v(kozel<\/strong>) + v(\u0159idi\u010d<\/strong>) najdeme jako odpov\u011b\u010f slovo \u0159idi\u010dka<\/strong> jednodu\u0161e proto, \u017ee se v bl\u00edzkosti \u017e\u00e1dn\u00e9 jin\u00e9 slovo nenach\u00e1z\u00ed.<\/p>\n","protected":false},"excerpt":{"rendered":"

Od publikov\u00e1n\u00ed prvn\u00edho algoritmu Tom\u00e1\u0161e Mikolova, p\u0159ev\u00e1d\u011bj\u00edc\u00edho slova na vektory se zaj\u00edmav\u00fdmi vlastnostmi (psal jsem o n\u011bm zde), ub\u011bhlo ji\u017e n\u011bkolik let. St\u00e1le se v\u0161ak lid\u00e9 sna\u017e\u00ed v\u00edce \u010di m\u00e9n\u011b uspokojiv\u011b vysv\u011btlit, jak je mo\u017en\u00e9, \u017ee \"kings \u2013 king + queen\u00a0 = queens\". P\u0159ed \u010dasem napsal podobn\u00fd text na sv\u00e9m blogu\u00a0Piotr Migdal. \u010cl\u00e1nek je inspirativn\u00ed, […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[10,14],"tags":[],"class_list":["post-328","post","type-post","status-publish","format-standard","hentry","category-nlp","category-strojove-uceni"],"_links":{"self":[{"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/posts\/328","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/comments?post=328"}],"version-history":[{"count":34,"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/posts\/328\/revisions"}],"predecessor-version":[{"id":452,"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/posts\/328\/revisions\/452"}],"wp:attachment":[{"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/media?parent=328"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/categories?post=328"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.mlguru.com\/cs\/wp-json\/wp\/v2\/tags?post=328"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}