This job posting isn't available in all website languages


For Nokia Internal Employee access Log in here

Join us at Nokia to connect the world

Search jobs

Cifre thesis (PhD): Cost and Performance optimization in cloud:combining model-based and model-free AI approaches

Applied R&D
BLCTO Bell Labs & CTO
1800000FP6 Requisition #
Apply for Job
Share this Job
Sign Up for Job Alerts
Cost and Performance optimization in cloud: combining model-based and model-free approaches


Growing digitalization is rapidly expanding the complexity and heterogeneity in computing systems such as Cloud computing. In these systems, a multitude of applications share the same infrastructure which includes diverse hardware and system software. In addition, different parties such as Cloud providers and Cloud users may be reluctant to share information with one another. Moreover, due to the resource sharing paradigm, the cloud characteristics can change dynamically from its tenant perspective. The swollen complexity creates a major challenge for system management and control. Traditional approaches to resource management based on system model specification, off-line behaviour learning and traffic prediction will be increasingly defied due to the emergence of these complex and dynamically evolving systems.
To deal with this challenge, a new paradigm of continuous learning in interaction brings a strong promise for highly adaptive control mechanisms. Starting with little or no knowledge about system characteristics, the control agents start taking actions and learn on-the-fly about their efficiency through the observed feedback from the environment. This approach, referred to as Reinforcement Learning (RL), allows to hide the inherent complexity of the environment and to adapt dynamically to its changing conditions. 
In this work, we seek to create and experiment with Reinforcement Learning algorithms which automatically adapt to unknown system specifics such as application traffic patterns, resource consumption rate, system capacity, or system’s input-output dynamics, and derive effective resource management and control policies.
We target complex applications and infrastructure where controls are exerted at multitude of places. We focus on the capacity management aspect and create Reinforcement Learning algorithms to decide when to use how much capacity at what place. Efficient decisions require trading off between the operating expenses of providing the capacity, including energy consumption, equipment repair and replacements, etc., and the service performance achieved, such as throughput, response time, etc.
We aim to create Reinforcement Learning algorithms that learn well for practical problems. These algorithms will learn rapidly in the incremental, online settings for many practical problems. They will also learn safely, limit the negative impact of random exploration actions and be capable to handle faults and failures common in practical problems. Moreover, they will learn in a distributed manner for complex and heterogeneous applications and infrastructure.
Current Reinforcement Learning algorithms struggle in these aspects. To overcome the challenges, we will derive structural properties of the practical problems from model-based approaches, such as Markov decision process, stochastic optimization, game theory, etc. We will exploit these properties to specialize Reinforcement Learning algorithms and speed up the learning. We will also use them to devise effective exploration strategies and incorporate information from other system functions such as fault detection and diagnosis. We will build upon them to develop efficient collaboration schemes so that the distributed learning achieves overall system efficiency. The model-free and model-based approaches can be combined to solve practical problems online. The model-free approach ensures the adaptability, while the model-based approach ensures fully utilization of experiences.


Duration: 3 years (full time).  


Ideal profile:
We look for students that have sound theoretical understanding of different approaches and are proficient in conducting numerical experiments on them. They should have a Master or Engineering degree in computer science, operations research, or machine learning. He or she will create a bridge between the respective Bell Labs and University of Paris-Sud teams.

Previous Job Searches

My Profile

Create and manage profiles for future opportunities.

Go to Profile

My Submissions

Track your opportunities.

My Submissions
Life at Nokia
Explore Employee Blogs
We create the technology to connect the world

Stay in touch with us through our social media channels:

Follow us on Facebook
Follow us on LinkedIn
See us on Glassdoor
Follow us on Twitter
Please try again.


Either there was a problem on our end with the action you just performed, or we are currently having technical difficulties with our system. Please try again later.

Teams at Nokia

See all jobs

Research & Development

See new jobs

Market & Sales development

See new jobs

Corporate services

See new jobs
Nokia is an equal opportunity employer that is committed to diversity and inclusion. At Nokia, employment decisions are made regardless of race, color, national or ethnic origin, religion, gender, sexual orientation, gender identity or expression, age, marital status, disability, protected veteran status or other characteristics protected by law.