Note: This conversation is heavily paraphrased. Apologies if I screw something up. And apologies for the "some might say" phrase. I just don't understand those positions well enough to either believe them or faithfully report them.
Yaeger: knowledge is represented by the strength of the connection.
Me: (In my head) Some might say knowledge isn't represented.
Yaeger: learning occurs by adjusting the synaptic strength.
Me: Can learning occur without adjusting the synaptic strength? In other words, could a static network get it wrong the first ten times, and then get it right the 11th?
Yaeger: (thinking computationally) No, if the network doesn't change, how can the behavior change?
Me: I'm thinking about a living animal, not a dead one. Lingering patterns of activation could change the behavior of the network.
Yaeger: (and this is HEAVILY paraphrased) OK, so patterns of activation might have some effect on the behavior of the network, but how could you guarantee that it would learn the right stuff?
Me: Well, I don't know. I'm just asking if it's possible that learning could occur only through activation.
I'll write something about this in Transitory Learning.
Complex systems can't be modeled... I don't understand why they can't do things turing machines (which are a model) can't do.
Sounds like a "holy fuck" moment. (Minsky & Papert 1969)
I should look it up
Yaeger: They kind of encapsulate how networks capture knowledge.
Yaeger: They like spatial things, they like blurry things.
Me: (In my head) How does this relate to perception/action? Do we need these kinds of inputs because we are misinterpreting the nature of neural "input"?
Me: (In my head) The input/output model seems like a biological violation. The brain doesn't seem to have nice neat components with inputs and outputs, it seems to be a giant mess. I don't really know enough about it to have a formal objection though.
Yaeger: Representation (how you represent the training data) is really important.
Me: (In my head) Some might say that there are no such things as representations. The Robot Baby 2001 paper, for example, shows how we can learn to recognize objects from unstructured sensory input.
Yaeger: Learning in neural networks is minimizing error.
Me: (In my head) It seems like the big issue for this class for me is organizational dynamics. Are the dynamics just consequents of structure and physics, or is there a third thing that needs to be accounted for? Like inertia.