sirosh at cs.utexas.edu (Joseph Sirosh) writes:
JS> Sigmoidal activation functions have been used in many models of neurons.
JS> Some models allow this activation function to change, from flatter
JS> sigmoids to steeper or vice versa. Is this biologically plausible?
It is more than just plausible: it happens. There are at least half
a dozen physiological mechanisms for doing this, and probably many
more. To name just one, the neurotransmitter norepinephrine acts, in
some places, to reduce the firing rate for low amounts of input but
increase the rate for high amounts of input. This effect is sometimes
described as "increasing the signal-to-noise ratio of the cell."
JS> Change in activation function of a neuron can happen in many ways. One
JS> of them would be migration of ion channels over the surface of
JS> the neuron. [ . . . ]
No doubt this sort of thing happens, but it would be pretty slow. It
would probably take days to get a substantial change in the activation
function with a mechanism like this, at least for reasonable kinds of
stimulation.
JS> I have come across evidence that ion channels migrate during
JS> synapse formation at the neuromuscular junction( Kandel, Schwartz,
JS> Jessel - Principles of Neuroscience ). However, does similar
JS> migration of ion channels happen in normal neurons?
Yes, it must -- though no other system has been studied to nearly
the same extent as the neuromuscular junction. Even so, migration has
actually been observed in a couple of other places.
JS> In particular, is there any evidence that migration of ion
JS> channels is responsible for phenomena like habituation?
Very unlikely. The time scale is wrong. Habituation occurs in
seconds to minutes -- migration takes at minimum hours. (There are
some longer-term forms of adaptation that may be related to migration
of channels.)
JS> Is there evidence that the activation function changes when
JS> neurons are repeatedly stimulated? Say with a tetanus?
Yes. It can change in a variety of ways, depending on the nature of
the stimulus. This is one of the hottest topics in neuroscience now,
and the literature on it is so vast that it's hard to pick out a
single thing to point you to, but maybe a reasonable start would be
chapters 64 and 65 of Kandel, Schwartz, and Jessel.
JS> Is there evidence that neurons during development change from
JS> more or less linear and easily triggerable, to highly nonlinear
JS> as they mature?
The activation function certainly changes during development, but my
impression is that it tends to go in the opposite direction (i.e. from
less to more linear). A simple Hodgkin-Huxley spiking mechanism is
extremely nonlinear (that is, the sigmoid is very steep, almost like a
step function). Special kinds of ion channels have to be added in
order to stretch out the "linear" region. But there is unlikely to be
any universal rule for the changes that occur during development.
-- Bill
{Why does this remind me of my prelims?! :-) }
