I have mostly taken courses and done teaching during my Ph.D. so far. But this semester, I have focused on research. The idea is to use so-called multi-armed bandits to improve Brain-Computer Interfaces (BCI).
Multi-armed bandits
The challenge in a multi-armed bandit problem is to choose an action that gives you the highest reward. For example, you have moved to a small town and want to find the best of the five restaurants. Assuming that you have no friends to ask and can’t google for reviews, you would choose a restaurant at random and give it some rating. The next day, you would choose another restaurant and give it some rating until you have tried all restaurants. Then, you would always visit the one with the highest rating. But what if you had bad luck and got a dish that you did not like at one of the restaurants and gave it an unfairly low rating? Because of this risk, it is worth visiting all restaurants more times to ensure that you have a fair rating. Hence, you need to explore all available actions (restaurants) before exploiting the best one. The multi-armed bandit problem is the simplest form of reinforcement learning and is one approach to solve the exploration vs. exploitation problem.
The research idea
One problem with BCI systems is that they need calibration before using them. The calibration takes a lot of time, is boring, and tiresome for the user. If the BCI system could use data from previous sessions for calibration, it would reduce the calibration time. My supervisors and I believe that BCI systems can use the multi-armed bandit formulation to find similar data between users. Instead of finding the best restaurant, the task is to find the best data.
Writing a paper
Preparing for the research idea, I have read a lot of papers – both about multi-armed bandits and BCI+multi-armed bandits. I have come to realize that there are no good summaries that introduce multi-armed bandits for BCI researchers. So before investigating our research idea, I have started to write a review on Multi-armed bandits for BCI systems. Apparently, I like to write because it has been surprisingly fun to write the paper. I now have a finished draft, and the next step is to decide on a journal to submit it to. This paper is my first, and I have often felt like a newborn giraffe trying to stand. I have no idea how one usually writes a paper or where to submit it, but somehow I have managed.
There is still a lot of work to be done before submitting, but I’m very proud of myself so far!
Frida Heskebeck 