Question: is there a neural network mechanism for "figuring things out"? This is getting a little new-agey, but what I'm really asking about is a NN mechanism for creativity. So, when I look at something and try to figure out what it is... if I fail, I don't just stop there, I try to be creative. I try transformations and other interpretations of the same input. It seems like this would *have* to be using activation of some sort.
So I guess the suggestion is: temporary reasoning wouldn't happen in the weights, and there's some "learning" involved in this.
We talked about the criticism that the NN approach amounts to behaviorism. Yaeger was like: No way! Behaviorists deny the existence of representations, and we NN people talk about representations all the time. He seemed to suggest that the non-representational approach was just silly.
This made it very clear that taking the distributed approach doesn't commit you to either side of the cognitive/ecological debate. I think you see something like this:
The ecological people don't believe in representations themselves, so they don't care if networks don't have them.
Question for ecological people: Are invariances really never represented? Is the argument that a representation would have to be the entire system, so it is meaningless?
Yaeger: "macro-theories should evolve from micro-theories"
We discussed this to death. I probably don't understand complex systems well enough to invoke them as often as I do.
Yaeger: (paraphrasing) the argument that you need a "language" for every level of behavior is just not relevant. Understanding emergent properties requires a theory of underlying interactions.
Me: This runs very contrary to my intuition about emergent properties. My understanding is that you can watch the behavior of small bits of the system and see how the rules play out, but that the effects of these behaviors sort of disappear into the system, and then emerge from that behavior. So if you have two emergent phenomena X and B, a portion of the system A, and a rule K, you can say:
But because B is itself an emergent phenomenon, you don't really have an account for X. And you can't just apply this sort of analysis recursively, because complex systems just don't reduce like that. Am I wrong?
Ecological Psychology as a Basis for Artificial Cognition - this is a Frankenstein of a topic. But it popped into my head as a reasonably approximation of what I'm trying to do. I think that there are some low-level properties of ecological psychology that enable intelligent behavior. I'm interested in applying them to artificial systems in the same way that ALife people apply natural selection. I think that would be a way of studying the broader class of cognitive phenomena (not just in people, but all possible forms of cognition). I'm probably missing something important.
BTW, I really do think this is HCI.
Musings:
who is doing an ecological approach to neural nets?
are connections in dentrites themselves computational gates of some sort?
what is brain damage due to inactivity? does it tell us anything about learning in the weights?