If you have lots of affiliate links on your website, does the page rank go down or the serp

decrease?

As i understand it no, the links that are on your site shouldnt affect you search engine placement, however if you are using excessive repetitive links you may run into problems that the search engines will see your site as spamming them with unnesasary backlinks (links to other sites).

Typically, you don't want to have a page that is just a long list of links. As Joel said, this may be viewed as "spamming". It's a better practice to give some sort of description of each link, and use keywords for that link in the "anchor text" (the text that is actually clicked on for the link). Also, you may be penalized for linking to a website that is predominately links. The search engines are really cracking down on these "link farms", and anyone who links to them. Google is aggresively spidering sites with two different crawlers now. There is much speculation as to why. One theory is that they are really going after "spam" techniques, but no one really knows for sure.