I think you didn’t understand what I was trying to say.
On the first portfolio page, you probably have only the items which are from category 1,2 or what ever. You do not have any items which are from third category.
Try to display 24-30 items on main page and see if you have the same problem.
The filter option only filters the items from the current page, not from all 18 pages.