Sarah
New Member
Posts: 31
|
Post by Sarah on Aug 30, 2017 23:05:20 GMT 8
Any ideas why there would be a gap in the distribution of stars in the flux vs. flux_err plane (both from segmentation statistics of ProFound)? It's clearly visible at the same place in all of my KiDS optical bands, though it's most pronounced in the r-band (which is the deepest one). It shifts to lower values of flux/flux_err and pretty much disappears when setting skycut to a lower value of 1.  
|
|
asgr
Junior Member

Posts: 56
|
Post by asgr on Aug 31, 2017 16:51:43 GMT 8
At a slight guess, those bright objects are likely to be stars, and they tend to have a much higher mean S/N. To be sure try comparing to the surface brightness or size of the objects.
A
|
|
Sarah
New Member
Posts: 31
|
Post by Sarah on Aug 31, 2017 23:20:45 GMT 8
All of those dots are stars (across many cutouts around galaxies in many different KiDS tiles); or to be more precise they passed my star selection criteria, which are based on R50 and con (mainly, though I also limit the magnitude range, axial ratio and check for isolation). I couldn't see anything obvious in the S/N vs. R50 distribution:  Red dots here are the stars in a particular example cutout which I use below to see where my stars sit relative to other objects (I don't have that information saved for all cutouts used above). So these two plots below show the distribution of all objects in the example cutout (green dots) and selected stars (red) again in the S/N vs. R50 plane (as above) and also in surface brightness vs. R50.   Note the star selection doesn't aim to find all stars but to find a set of stars suitable for PSF fitting for my purposes (so e.g. the brightest objects are excluded deliberately).
|
|
|
Post by Aaron on Sept 1, 2017 20:09:17 GMT 8
Just wondering- could this be something to do with variable stacking depth in the data?
A
|
|
Sarah
New Member
Posts: 31
|
Post by Sarah on Sept 2, 2017 0:05:25 GMT 8
But if it were stacking depth I'd have expected the gap to be at the same value of S/N for different choices of skycut; and different for bands of different depths. Instead it's the same across bands but moves with skycut. It might be something odd about my star selection, I'll have to look at that in more detail...
|
|
Sarah
New Member
Posts: 31
|
Post by Sarah on Sept 4, 2017 21:48:11 GMT 8
I did some quick tests with made-up data, see code and plots below. All stars are the same here except for their magnitudes. Star parameters, skyRMS, gain and magnitude range are "inspired" by KiDS r-band data (although obviously the magnitude distribution of stars etc. isn't very realistic). The point is that these plots show the same strange behaviour (even more pronounced since less noisy) as the real data, so I'm now suspecting there's something odd going on with the flux_err estimation (rather than with my star selection). flux_err is dominated by flux_err_skyRMS; flux_err_sky is a few %-level effect while flux_err_shot is negligible. The same thing happens if I don't pass a sky, skyRMS, or gain to ProFound.   library(ProFit) library(ProFound) skyc=2 Nstar=100 loopno=30 imagedim=500 segstats=data.frame() for(i in 1:loopno){ print(i) starmodel=list( moffat=list( xcen=runif(Nstar, 0, imagedim), ycen=runif(Nstar, 0, imagedim), mag=seq(19, 23, length.out=Nstar), fwhm=rep(3, Nstar), con=rep(2.1, Nstar), axrat=rep(1, Nstar), ang=rep(0, Nstar), box=rep(0, Nstar) ) ) starim=profitMakeModel(modellist=starmodel, dim=rep(imagedim,2))$z testim=starim+rnorm(length(starim), mean=0, sd=4e-12) testseg=profoundProFound(testim, skycut=skyc, sky=0, skyRMS=4e-12, gain=1.6e13) segstats=rbind(segstats, testseg$segstats) } magplot(segstats$mag, segstats$flux/segstats$flux_err, xlab='magnitude', ylab='S/N (flux/flux_err)', col=rgb(0,0,1,0.5), pch=19, cex=0.2, main=skyc) magplot(segstats$flux, segstats$flux_err, xlab='flux', ylab='flux_err', col=rgb(0,0,1,0.5), pch=19, cex=0.2, main=skyc)
|
|
asgr
Junior Member

Posts: 56
|
Post by asgr on Sept 6, 2017 13:15:02 GMT 8
Hmm, that's very useful Sarah. I will reproduce this and have a dig around.
A
|
|
asgr
Junior Member

Posts: 56
|
Post by asgr on Sept 6, 2017 13:30:46 GMT 8
Thanks to those plots I think I've sussed out what you are seeing here. In short: there's nothing really wrong, rather it is a natural feature of the dilation process for PSFs.
The plots below tell the full story. As you say the skyRMS mostly dominates the error (as we would expect for all but the brightest sources). This is well predicted by N100, and N100 for stars (PSFs) is sensitive to the number of dilations before convergence is reached. For fainter PSF sources more dilations are required on average, meaning larger apertures and larger errors (naturally enough). Since the dilations are a discrete process this produces the visual artefacts you pointed out. I think that solves the mystery. They are to be expected basically, and I don't think there is reason to be too concerned.
magplot(segstats$flux, segstats$flux_err_skyRMS, xlab='flux', ylab='flux_err_skyRMS', col=rgb(0,0,1,0.5), pch=19, cex=0.2, main=skyc, log='xy') magplot(segstats$iter, segstats$flux_err_skyRMS, xlab='iter', ylab='flux_err_skyRMS', col=rgb(0,0,1,0.5), pch=19, cex=0.2, main=skyc, log='xy') magplot(segstats$N100, segstats$flux_err_skyRMS, xlab='N100', ylab='flux_err_skyRMS', col=rgb(0,0,1,0.5), pch=19, cex=0.2, main=skyc, log='xy')
magplot(segstats$iter, segstats$N100, xlab='iter', ylab='N100', col=rgb(0,0,1,0.5), pch=19, cex=0.2, main=skyc, log='xy') magplot(segstats$iter, segstats$R50, xlab='iter', ylab='R50', col=rgb(0,0,1,0.5), pch=19, cex=0.2, main=skyc, log='xy') magplot(segstats$iter, segstats$R100, xlab='iter', ylab='R100', col=rgb(0,0,1,0.5), pch=19, cex=0.2, main=skyc, log='xy')
Is there anything else to be worried about on your side, or was it more about understanding the origin?
Cheers,
A
|
|
asgr
Junior Member

Posts: 56
|
Post by asgr on Sept 6, 2017 13:36:39 GMT 8
Oh, and the reason I added the R50 plot was to show that it is much more stable against dilation artefacts, barely evolving with iteration number at all. You might want to put a limit on the allowed number of dilations: real stars should not need many 2-3, so you could consider changing the iter limit to 3 rather than using the default of 6 (more appropriate for galaxies, which is what I tuned the default with).
A
|
|
Sarah
New Member
Posts: 31
|
Post by Sarah on Sept 6, 2017 15:51:01 GMT 8
Ah OK, that's good to know, thanks. It was more about understanding the origin really, essentially I was worried there might be some weird bias in my star selection, so it's good to know that it comes from somewhere else and is understood. Since I'm mainly using R50 then I guess it won't matter too much to me. It could also be an explanation for why I see a range of concentrations for stars of a given R50 (which is something else I've been wondering about): Basically, R90 depends on the iteration number while R50 doesn't, so for a given R50 there's going to be a range of con corresponding to the range of iteration numbers. R90 tends to go down with iteration number, so con goes up; so higher concentration means fainter stars on average (with higher iter), which is also what I observed. Does that sound about right?
|
|
|
Post by Aaron on Sept 7, 2017 12:04:55 GMT 8
Yes, that's a good point. You might want to select by number of iterations too- e.g. only objects with exactly 2.
Over here we have chosen not to use concentration to do the separation: R50 v SB90 v axrat is out preferred single band split.
A
|
|