A blind test of photometric redshifts on ground-based data

Aims. Several photometric redshift (photo-z) codes are discussed in the literature and some are publicly available to be used by the community. We analyse the relative performance of different codes in blind applications to ground-based data. In particular, we study how the choice of the code-template combination, the depth of the data, and the filter set influences the photo-z accuracy. Methods. We performed a blind test of different photo-z codes on imaging datasets with different depths and filter coverages and compared the results to large spectroscopic catalogues. We analysed the photo-z error behaviour to select cleaner subsamples with more secure photo-z estimates. We consider Hyperz, BPZ, and the code used in the CADIS, COMBO-17, and HIROCS surveys. Results. The photo-z error estimates of the three codes do not correlate tightly with the accuracy of the photo-z's. While very large errors sometimes indicate a true catastrophic photo-z failure, smaller errors are usually not meaningful. For any given dataset, we find significant differences in redshift accuracy and outlier rates between the different codes when compared to spectroscopic redshifts. However, different codes excel in dfferent regimes. The agreement between different sets of photo-z's is better for the subsample with secure spectroscopic redshifts than for the whole catalogue. Outlier rates in the latter are typically larger by at least a factor of two. Conclusions. Running today's photo-z codes on well-calibrated ground-based data can lead to reasonable accuracy. The actual performance on a given dataset is largely dependent on the template choice and on realistic instrumental response curves. The photo-z error estimation of today's codes from the probability density function is not reliable, and reported errors do not correlate tightly with accuracy. It would be desirable to improve this aspect for future applications so as to get a better handle on rejecting objects with grossly inaccurate photo-z's. The secure spectroscopic subsamples commonly used for assessments of photo-z accuracy may be biased toward objects for which the photo-z's are easier to estimate than for a complete flux-limited sample, resulting in very optimistic estimates.