What if you add an extra button on the right side bar and when you click it it just shows all the comments in the conversation.

I would not remove the comments from the conversation view. I would leave them there because like mentioned in the past its better to have them there because it gives you context for what the comment was about.

The idea I had was just list all comments in the order they were entered so you can look through them and find something from way back.

We have orders that span 3 months and have lots of emails back and forth and comments so you can get caught scrolling back a ways through the conversations looking for comments.