IUBio

NN formats

Jiri Donat jiri_donat at hotmail.com
Tue May 25 03:50:05 EST 1999


In article <37408107 at news3.us.ibm.net>,
  "Sergio Navega" <snavega at ibm.net> wrote:
> Patrick Juola wrote in message <7hpk1p$u4$1 at quine.mathcs.duq.edu>...
> >In article <3740344e at news3.us.ibm.net>, Sergio Navega
<snavega at ibm.net>
> wrote:
> >>Jiri Donat wrote in message <7hoift$1ie$1 at nnrp1.deja.com>...
> >>>
> >>>To me, the biggest difference between natural NN and ANN is that
every
> >>>digital simulation of ANN network has a discrete set of states
(however
> >>>large the set is). This "limitation" (if we understand this
feature of
> >>>digital representations of ANNs on today's computers as a
limitation -
> >>>and some theories do) is inherited in our existing tools for ANN
> >>>simulations - in digital computers.
> >>>
> >>
> >>
> >>I'm not sure I understand you here. In fact, biological neurons
> >>seen from "outside" are just things that fire or don't fire, in a
> >>purely discrete manner.

Yes, I originally meant the hidden complexity of neurone (the neurone
from the inside point of view), which is just roughly approximated by
our current ANN models. In other words, "neurone" in the ANN meaning is
just a label (unprecisely) used for description of the basic building
block of ANNs.

This viewpoint is taken as the basis of some theories which then
speculates about the level of significance of neurone in comparison
with the significance of the Neuronal network. So, these theories to
certain extent reverts the classical view that it is the network which
is responsible for the complexity and in which construction simple
building blocks (neurones) are used. (Personally, I do not consider
this question being important - the inherent complexity of Nature gives
us a sort of third view; putting it simply, we can find complexity and
answers to our questions in every object of Nature we choose to
investigate. And in our level of understanding it is certainly more
practical to investigate NNs instead of neurones).

> >
> >Seen from *FAR* outside, yes.  In a similar fashion, from
sufficiently
> >far outside, the Solar System is a point mass.
> >
>
> My point exactly. If we're studying the kinematic behavior of the
> milky way, it doesn't pay off to know the mass of Mars.
>
> >> There don't seem to be any other meaningful
> >>characteristic (such as waveshape or voltage) from the output of a
> >>biological neuron, just the presence or not of the pulse.

Now we look to the neurone from the outside point of view. We should
distinguish these two views. From the outside perspective the neurone
certainly looks pretty "digital", and so does the configuration of
natural NN. In other words, it looks like a digital code (or protocol)
is used in natural NNs. Decoding such an protocol would be a great
achievement in the whole area of NN and a good step forward in ANNs.
Are you aware of any progress made in this direction?

It seems to me that scienticts are busy at the moment observing just
the "physical layer" (if we use the OSI model which is known from
computer networks), e.g. the "firing rate". Till now, scientists are
not modelling the possible higher layers; only constructing such models
can lead to decoding the information carried.

Or we can put it in other, more simple analogy: let us compare the
firing of a neuron to a simple radio frequency modulation (a
traditional analogue FM broadcasting). This analogy can also help us to
understand how far is our current understanding of "traffic" in NN from
obtaining the first meaningful results (=from understanding the
internal "language" of the NN).
In FM modulation the "firing rate", or frequency, does not matter at
all (different stations use different frequencies - carriers). On the
other hand, changes of frequency are the important factor. By observing
changes over the time we can construct another frequency, which already
contains the information. Unfortunately, decoding this information is
another significant task. We should interpret this decoded (in
technical terms "demodulated") frequency as audio waves. These waves
carries the information, but the information itself is coded in natural
language (!).

Doesn't such a model lead to a certain level of scepticism?

> >
> >This is incorrect.  What's usually regarded as more important than
> >the presence or absence of a pulse is the firing *rate*, measured
> >as a scalar quantity.   Different neurons respond at different rates
> >depending on the circumstances, &c.
> >
>
> Although this is an open issue (not all neuroscientists agree with
> firing rate), your assertion does not invalidate what I've said.
> In other words, I said that each element of a spike train does
> not seem to be differentiable by such characteristics as waveshape or
> voltage, but only by the discrete presence or absence of the pulse
> (while composing a scalar, mean firing rate or by means of timing
> among each spike is something I didn't mention).

You are approaching the problem of constructing the language model of
NN internal communication here. As follows from what I mentioned above,
I agree with you.

>
> >Viewed in this light, the (scalar) activation level present at
> >the output of an ANN unit is a model of the scalar activation level
> >of the output of a real neuron.  In more sophisticated models, this
> >time course can be explicitly taken into account -- Birkbeck College,
> >Univ. of London and Cal-tech both have active research groups looking
> >at this sort of model.
> >
>
> I agree with that, but this is part of the open issue. Apparently
> what is being settled is that neurons close to sensory inputs seem
> to work considering the mean firing rate and that neurons of more
inner
> portions of the cortex care more for the "individual spikes". For
> instance, it has been demonstrated by Steveninck and Bialek that
> single spikes of the visual system of the fly contain significant
> information about the stimulus.
>
> But I think there's an additional contestant here, and this is
> related to the proposals that put a role on the synchrony of
> populations of neurons (Wolf Singer is an important name in this
> regard). So although all these results are not meant to dismiss
> current models of ANNs, I would certainly assume that what ANNs
> model is a very different kind of thing than biological neurons.

My hope comes from our day-to-day experience, from observations of
Nature: the natural communication tools (incl. the native language)
seems to be pretty redundant. So it still should make a good sense to
model the internal NN "language" used in the communication of
individual neurons; to "understand" (or interpret) this language we
should however know the context of this communication in bigger
neuronal populations (this is a parallel from comprehending a native
language - no one letter can be comprehended, even no one word; in
certain situations you need even more than one sentence to understand
the idea).
Saying that, it certainly makes less sense to investigate the details
of individual communication (waveshape or voltage). To me, it makes a
good sense to try to collect the communication bits to bigger groups
and try to apply some models of computer communication protocols.

I hope some work is going on in this direction already?

>
> Regards,
> Sergio Navega.
>
>
Best regards,
--
Jiri Donat (jiri at calresco.org)


--== Sent via Deja.com http://www.deja.com/ ==--
---Share what you know. Learn what you don't.---



More information about the Neur-sci mailing list

Send comments to us at biosci-help [At] net.bio.net