Why isn't it okay for me to counter someone's opinion?

Why isn't it okay for me to counter someone's opinion? Two co-workers are cyclists that ride bikes to work, and they are both express their "anti-vax" opinions regularly. Almost sounds like the start of a bad joke. One of them told me that when a driver pulled in front of them to get ahead of them in the right lane, (too close for their comfort) they rode past and smacked their passenger side mirror on purpose. I said "well, that's part of why drivers hate cyclists". I said I try to be respectful of cyclists (even though they irritate me) but you can't expect everyone to be the same. My city is not cyclist-friendly, and we work in an industrial area. Every other day one of them has a story about how a driver was too close to them. If I was a cyclist and was constantly worried about drivers, I wouldn't ride my bike to work. They both have vehicles, I don't get it? The other co worker keeps going on about how the vaccine is ineffective etc. We all had to be vaccinated to keep our jobs, which made me uncomfortable but its not in my control. At this point I'm pretty tired of hearing anything covid related, so I said can we knock it off with the covid/vaccine talk a few days ago. I was not as polite about it as I could have been, I will admit. But I certainly held back what I actually wanted to say. Why is it fair that I have to sit through your conspiracy theory talk regularly on my lunch break? Now after these two conversations, both co-workers avoid me. Why is it okay that people can go on about these types of things, and it's expected of everyone else to just... agree with them? Keep quiet about their own opinions? For the most part, I do my darndest to keep to myself and try hard to get along with everyone. But I suppose I reached a breaking point on these topics. These are just two examples of course, I have always had a difficult time with this type of thing. Does anyone else struggle with this type of thing?