If you are having issues running Cognos Report Studio then one item to validate is the settings for your pop-up blocking software on your computer. Starting Report Studio opens a new browser window and it also opens a new browser window when you run a report.
Check to see if disabling any pop-up blocking software on your PC eliminates the issue.