Searching for Activation Functions

arxiv:1710.05941

Original source · Paper JSON

LLMs: architectures, context, training · 2017-10-16

Cites (0) · Cited by (6)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2

Contents

• Searching for Activation Functions
• Abstract
• 1 Introduction
• 2 Methods
• 3 Search Findings
• 4 Swish
• 5 Experiments with Swish
• 5.1 Experimental Set Up
• 5.2 CIFAR
• 5.3 ImageNet
• 5.4 Machine Translation
• 6 Related Work
• 7 Conclusion
• Acknowledgements