Be careful with drug dealers, try to either just ignore them and carry on, or try and pretend you speak another language, but do not pressure them or wind them up as they then may get angry, try to stick to well-lit areas, and do your research before travelling anywhere unknown.
Like most of France, Toulouse is a very safe city, and you are unlikely to encounter any problems. Nevertheless, it is always wise to keep a close eye on your belongings and your surroundings.
Pay attention to pickpockets especially during the summer months, as in and around Toulouse can be very busy and especially with people visiting San Tropez which is near-by, pickpockets will be able to tell which are tourists and which are unsuspecting, as a precaution keep all valuables at home or at the hotel, and watch all belongings.