Back to Yann's Home
Publications
 
LeNet-5 Demos 

Unusual Patterns 
unusual styles 
weirdos 

Invariance 
translation (anim) 
scale (anim) 
rotation (anim) 
squeezing (anim) 
stroke width (anim) 

Noise Resistance 
noisy 3 and 6 
noisy 2 (anim) 
noisy 4 (anim) 

Multiple Character 
various stills 
dancing 00 (anim) 
dancing 384 (anim) 

Complex cases (anim) 
35 -> 53 
12 -> 4-> 21 
23 -> 32 
30 + noise 
31-51-57-61

LeNet-5, convolutional neural networks
Convolutional Neural Networks are are a special kind of multi-layer neural networks. Like almost every other neural networks they are trained with a version of the back-propagation algorithm. Where they differ is in the architecture.  
Convolutional Neural Networks are designed to recognize visual patterns directly from pixel images with minimal preprocessing. 
They can recognize patterns with extreme variability (such as handwritten characters), and with robustness to distortions and simple geometric transformations.  

LeNet-5 is our latest convolutional network designed for handwritten and machine-printed character recognition. 
Here is an example of LeNet-5 in action. 

 
 

Many more examples are available in the column on the left:

Several papers on LeNet and convolutional networks are available on my publication page:

[LeCun et al., 1998]
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, november 1998. Gzipped PostScript, 75 pages, 709897 bytes

[Bottou et al., 1997]
L. Bottou, Y. LeCun, and Y. Bengio. Global training of document processing systems using graph transformer networks. In Proc. of Computer Vision and Pattern Recognition, Puerto-Rico, 1997. IEEE. DjVu Gzipped PostScript, 6 pages, 61286 bytes

[LeCun et al., 1997]
Y. LeCun, L. Bottou, and Y. Bengio. Reading checks with graph transformer networks. In International Conference on Acoustics, Speech, and Signal Processing, volume 1, pages 151-154, Munich, 1997. IEEE. DjVu Gzipped PostScript, 4 pages, 49031 bytes

[LeCun and Bengio, 1995a]
Y. LeCun and Y. Bengio. Convolutional networks for images, speech, and time-series. In M. A. Arbib, editor, The Handbook of Brain Theory and Neural Networks. MIT Press, 1995. DjVu Gzipped PostScript, 14 pages, 40455 bytes

[LeCun et al., 1995a]
Y. LeCun, L. D. Jackel, L. Bottou, A. Brunot, C. Cortes, J. S. Denker, H. Drucker, I. Guyon, U. A. Muller, E. Sackinger, P. Simard, and V. Vapnik. Comparison of learning algorithms for handwritten digit recognition. In F. Fogelman and P. Gallinari, editors, International Conference on Artificial Neural Networks, pages 53-60, Paris, 1995. EC2 & Cie. DjVu Gzipped PostScript, 9 pages, 46834 bytes

[Vaillant et al., 1994]
R. Vaillant, C. Monrocq, and Y. LeCun. Original approach for the localisation of objects in images. IEE Proc on Vision, Image, and Signal Processing, 141(4):245-250, August 1994. DjVu Gzipped PostScript, 16 pages, 415287 bytes

[Matan et al., 1992b]
Ofer Matan, Christopher J. C. Burges, Yann LeCun, and John S. Denker. Multi-digit recognition using a space displacement neural network. In J. M. Moody, S. J. Hanson, and R. P. Lippman, editors, Neural Information Processing Systems, volume 4. Morgan Kaufmann Publishers, San Mateo, CA, 1992. DjVu Gzipped PostScript, 1 pages, 51438 bytes

[Boser et al., 1991]
B. Boser, E. Sackinger, J. Bromley, Y. LeCun, and L. Jackel. An analog neural network processor with programmable topology. IEEE Journal of Solid-State Circuits, 26(12):2017-2025, December 1991.

[LeCun et al., 1990b]
Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. Handwritten digit recognition with a back-propagation network. In David Touretzky, editor, Advances in Neural Information Processing Systems 2 (NIPS*89), Denver, CO, 1990. Morgan Kaufman. DjVu Gzipped PostScript, 9 pages, 83969 bytes

[LeCun, 1989b]
Y. LeCun. Generalization and network design strategies. Technical Report CRG-TR-89-4, Department of Computer Science, University of Toronto, 1989.

[LeCun et al., 1989a]
Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1(4):541-551, Winter 1989.

[LeCun et al., 1989d]
Y. LeCun, L. D. Jackel, B. Boser, J. S. Denker, H. P. Graf, I. Guyon, D. Henderson, R. E. Howard, and W. Hubbard. Handwritten digit recognition: Applications of neural net chips and automatic learning. IEEE Communication, pages 41-46, November 1989. invited paper.

Yann LeCun