Future Work

As there are already working traffic lights in the simulation, an obvious next step is to have the agents successfully control the acceleration of the car. There would then be two neural networks: one for the steering and one for the acceleration, both working independently. It may be that a third neural network is required that classifies the colour of the lights, which would need to ignore the lights in the distance, and perhaps even the lights in front of it until the agent was close enough. While it may be possible to combine this with some kind of rule-based system (e.g. "if lights = red then stop") it may be better to feed the result of this network into the acceleration network. This is because acceleration depends on more than just the traffic lights; it depends on whether there are corners, obstacles etc, which would make it difficult or impossible to combine with rules. It may be that more networks need to be chained together in this fashion to achieve optimal performance. Having specialised networks chained together has the advantage of allowing each part of the chain to be trained independently, for example once the network had learnt to classify the traffic light colours correctly, there would be no danger of "untraining" it when training it to do something else, and it should also be easier for it to learn a few, separate concepts than one complicated concept.

Another important step would be to change it so the scene from the agent’s point of view was rendered into its own off-screen buffer, allowing multiple agents to interact at once. The agents would need to avoid each other, and it could be a novel approach to crowd simulation. Rendering the scene from each of the agents' points of view may prove to be too computationally expensive however, so it would be interesting to see how many agents could run at once, with performance be improved x-fold by having each agent's point of view being updated only once every x frames.

In [2], Reynolds identified three rules that can be followed in order to simulate flocking behaviour seen in animals such as birds. These rules are "separation" (i.e. moving to avoid collisions with others), "alignment" (i.e. moving in approximately the same direction as the others who are close), and "cohesion" (i.e. staying close to the others). It would be interesting to see whether flocking could be implemented in the system by having the trainer follow those three rules rather than explicitly programming them in. To achieve this, the set-up of the agent's point of view would have to be looked at carefully as animals that flock, such as birds, have their eyes located at the sides of their head to give a larger field of vision, compared to the relative tunnel vision that is currently used in the simulation.

There are many other possibilities that could be explored. For example, can a car learn to indicate before it starts turning? This would require the network to know that it was going to turn before it did. Or, can one car learn to follow another? Would it get confused when the car it was following went close to other cars? This problem may further expose the problem of the network having no memory, because if the car being followed managed to escape the field of view of the follower for even one frame, then the follower would have no idea where to go.

Perhaps the most important future work then is to investigate which problems can be solved by a neural network, and which are better to be solved using other methods.