Two Phase $Q-$learning for Bidding-based Vehicle Sharing

We consider the one-way vehicle sharing systems where customers can pick a car at one station and drop it off at another (e.g., Zipcar, Car2Go). We aim to optimize the distribution of cars, and quality of service, by pricing rentals appropriately. However, with highly uncertain demands and other uncertain parameters (e.g., pick-up and drop-off location, time, duration), pricing each individual rental becomes prohibitively difficult. As a first step towards overcoming this difficulty, we propose a bidding approach inspired from auctions, and reminiscent of Priceline or Hotwire. In contrast to current car-sharing systems, the operator does not set prices. Instead, customers submit bids and the operator decides to rent or not. The operator can even accept negative bids to motivate drivers to rebalance available cars in unpopular routes. We model the operator's sequential decision problem as a \emph{constrained Markov decision problem} (CMDP), whose exact solution can be found by solving a sequence of stochastic shortest path problems in real-time. We propose a novel two phase $Q$-learning algorithm to solve the CMDP.