Lewd comments and over the top remarks are usually how racism and sexism first begin creeping into the workplace. It all starts with subtle references which might initially be open to interpretation – “perhaps he didn’t really mean what I think he meant”, “maybe I misunderstood that reference” – and you shrug aside those comments your gut knows are racist or sexist in nature in the hopes that it was just a misunderstanding and probably doesn’t mean anything at all.

Truth is, however, it only gets worse from here on out. These offensive coworkers may just start ramping up their A-game and take their bullying to the next level. Companies have human resources departments for a reason – to handle employee grievances whenever (and by whomever) they occur, and HR managers should be taking the initiative to restrain all forms of bullying from taking place within the office premises. It becomes even more important when the nature of the bullying contains sexist and racist overtones.

According to a poll carried out by the Kapor Center for Social Impact – taken by 2000 US adults that left a technology related function – a toxic workplace culture is costing the industry more than $16 billion annually because they could not retain underrepresented talent. 1 in 10 women experienced sexism and 40% of men of color left because they felt they were mistreated.

When companies are unable to handle the toxic work environment in their office, it has the tendency to spill over to their public image and the rest of the world soon starts becoming privy of what’s going on inside the office. For small companies that never really had a strong footing to stand on in the first place, this can mean an early death for their business. If big companies like Uber and Google cannot handle the public backlash that comes with racism and sexism, smaller companies should definitely tread carefully.

The best way to deal with bullying at the office? Having a robust, efficient human resources team that is willing to step up when the need arises. This HR team should be just and impartial for all employees, regardless of their rank, power and influence within the company. They should be able to understand the grievances and concerns of the office staff and deal away with them before they turn into one giant problem for the whole workforce later on.

Your HR team will have to design reliable feedback mechanisms which allow the office culture to talk about and resolve difficult issues without the fear of facing any backlash from the higher ups. In this case, the boss should make it the order of the day that justice prevails and the violator of the company’s vision undergo proper disciplinary behavior and correctional measurements, and if necessary, even be let go.

Let’s face it – no matter how much an employee commits to the bottom line of your firm, if they’re turning the office into a hellhole for the underrepresented, your valuable resources are being forced to leave. It won’t really be long before an expensive and potentially damaging lawsuit comes knocking on your door either.