The little sine wave that could: abusing neural networks for fun and profit
Back when I was learning about neural networks as function fitters, I wondered if it is possible to force them to behave in strange ways for certain input values, as hidden features or easter eggs, like in video games. Let me illustrate what I mean: Imagine having a neural network trained at the simple task of calculating $y = sin(x), for~x \in [-\pi, \pi]$ Since neural networks are nothing more than universal approximators Ian Goodfellow et al. Deep learning, 2016; www.deeplearningbook.org Ian Goodfellow et al. Deep learning, 2016; www.deeplearningbook.org this task should be a breeze, even for a simple architecture. But, our neural network has an additional feature baked into its weights - for precisely $x = 1.2345$ it will spit out a bogus value, like $y=4.2$, which does not make any sense for a sine wave! ...