今日头条科学家李磊：会思考的通用智能机器还有多远？

2020-02-27 58浏览

  • 1.L ! .
  • 2.• • •
  • 3.• •e ArtificialIntelligence:a Modern Approach [Russell & Norvig]
  • 4.Can Computers Think? A question raised by Alan Turing in “Computing Machinery and Intelligence”, 1950
  • 5.• •e ArtificialIntelligence:a Modern Approach [Russell & Norvig]
  • 6.•f • • •2 • • •
  • 7.•d •
  • 8.Better than human in GO playing Via deep reinforcement learning and Monte-Carlo tree search
  • 9.Towards human level in Speech recognition Via end-to-end deep learning models Error rate 25 Speech Is 3x Faster than 20 Typing for English and 15 Mandarin Text Entry on 10 5 By 2015.12 DS2 Human Mobile Devices, with 20% - 50% lower 0 error rates. [Ruan et al, 2016] Deep Speech 2 [Amodei et. al. 2015]
  • 10.Telling stories in images A giraffe standing next to forest. [Mao et al 2015] [Karpathy et al, 2015] [Kiros et al 2015] [Vinyals et al 2015] [Chen & Zitnick, 2015]
  • 11.ToutiaoXiaomingbot:Automatic Olympic News Writing 16 days 450 articles 1million readers
  • 12.DL algorithms work well for Supervised learning data X !(⋅) label Y ./ Cat/dog/… Today is a nice day A giraffe standing next to forest ./
  • 13.Neuralnetworks:massively connected simple units Inspired by a biological neuron Imagecredit:http://cs231n.github.io/neural-networks-1/
  • 14.How to model a single artificial neuron? input x1 weight w1 Transfer function x2 w2 w3 x3 % Activation function
  • 15.Supervised Learning with Neural Nets h Input x 4 y=softmax(w2Ÿh1+b2) h1=tanh(w1Ÿx+b1)
  • 16.Deep Neural Nets Input h1=σ1(w1Ÿx+b1) h2=σ2(w2Ÿh1+b2) 16
  • 17.Casual chat generated by neural networks i So happy to have delicious lunch today! I want to eat too! t Let us befriend, rich guy ? s I am not rich Would you prefer falling in love at first sight or developing love over time? Falling in love at first sight probably Is Star Wars worth watching? Not very much [Fei Jiang & Lei Li 2015]
  • 18.What about longer utterance? a I once let the truest love slip away from r before my eyes, Only to find myself regretting when it L c was too late, No pain in the world comes near to this, If only God would give me another chance, I would say to the girl, I love you! If there had to be a limit of time, I pray it's ten thousand years. Bless you. 18
  • 19.Generating sentence with RNN Nonlinear dynamical system ht=σ(wŸht-1+uŸxt)Alternatives:LSTM (long-short-term-memory) GRU (gated recurrent units) softmax embedding 19
  • 20.Generating Emotional Responses embedding RNN Emotion excitation RNN softmax embedding Related work [Sutskever et al. 2014] [Bahdanau et al. 2015] [Shang et al. 2015] [Vinyals & Le. 220015]
  • 21.Answering Knowledge Questions subject string Who wrote Three Body Problem? subject relation object related work [Berant 2013] [Yih 2014] [Bordes 2015]
  • 22.Why difficult for machines? Language complexity Ambiguity Sparse Label • • • •
  • 23.Deep learning basedsystem:CFO u u o o - n 90.0% 70.0% 50.0% 30.0% 10.0% -10.0% Accuracy 56.0% 62.9% 75.7% [Yih et al, ACL 14] [Yih et al, ACL 15] [Bordes et al 2015]CFO:Conditional Focused Neural Question Answering with Large-scale Knowledge Bases [Zihang Dai, Lei Li, Wei Xu, ACL 2016]
  • 24.AI(ML) is more than supervised learning, and more than deep learning
  • 25.AI replies on Knowledge Representation Probabilistic Graphical Models Deep Learning Kernel Methods Sparse Representation Trees
  • 26.Limitations of exiting DL algorithms • Relying on huge labeled data • Generalizability / adaptability
  • 27.• (self-) interpretability • knowing why succeed/fail • Reasoning about objects in the world • Intelligence power density (per joule)
  • 28.Thanks! Contact lileilab@toutiao.com ! . 28