Remote Access

Until recently, I did not have a firewall installed on my computer. I know it is possible that a hacker could have accessed my computer and used it to send emails. There seem, however, to be a number of signs that my actual computer was used, and not just my email or ip address. For example, there is a record of someone searching for phone sex and chat lines on google and craigslist, and an email that was stored in the sent message folder on my hard drive (not accessible on-line). Is it possible that a hacker would/could leave those traces on my actual computer through the internet, or do I need to be concerned that someone physically broke into my room and sat down at my desktop? If a hacker could, WHY would s/he? (maybe that's beyond your purview...)
Thank you!