Gentoo penguins Pygoscelis papua breed at a number of sites at the West Antarctic Peninsula, including Goudier Island, Port Lockroy—the longest studied location for tourist-penguin interactions in the Antarctic. These penguins annually encounter some of the highest numbers of tourists in the whole of Antarctica. Using yearly count data from all ten colonies on Goudier Island, we report changes in long-term population size and breeding success over a 21 year period (1996/1997 to 2016/2017), documenting inter-annual variability in numbers of breeding pairs and chicks fledged (productivity). We found a 24.5% (1.4% per annum) decrease in breeding pairs; similar declines were evident in breeding pairs at six colonies visited by tourists as well as at four unvisited colonies. Breeding success also declined, with chick numbers declining in visited (53.7%, 3.8% per annum) and unvisited colonies (59.8%, 4.6% per annum). While gentoo penguin numbers are increasing regionally, we reveal a recent decline in the Goudier Island population occurring simultaneous with increases in tourist numbers from 262 in 1996/1997 to 19,688 in 2016/2017. Analyses suggest a complex situation with different possible drivers of change. There was a significant negative effect of increasing air temperature and shifts in sea ice variables on breeding pairs. However, similar statistical support showed a significant link existed between year and visitors, with higher numbers of visitors negatively affecting penguin numbers. Based on our results we recommend increased precaution in management at Goudier Island, and initiation of similar studies at other frequently visited penguin sites in Antarctica.