Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stardice dev #159

Open
wants to merge 128 commits into
base: stardice
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
128 commits
Select commit Hold shift + click to select a range
11430a0
Merge pull request #142 from LSSTDESC/v3.0.3
jeremyneveu Dec 19, 2023
3d0558e
Merge pull request #147 from LSSTDESC/v3.0.3
jeremyneveu Dec 20, 2023
c85d82f
typo
Dec 23, 2023
89e01c6
add flat and star_field attributes
Jan 23, 2024
0974e3d
rename stat_errors and stat_errors_rotated attributes in err and err_…
Jan 23, 2024
e107961
rename spectrogram in spectrogram_data
Jan 23, 2024
54bccd8
rename spectrogram in spectrogram_data
Jan 23, 2024
7fd2301
remove right_edge option
Jan 23, 2024
de7b913
initialize spectrogram starfield and flat
Jan 23, 2024
8157815
initialize spectrogram starfield and flat
Jan 23, 2024
2837c60
cleaning
Jan 23, 2024
c297b34
cleaning
Jan 23, 2024
a41a3a5
robust ndof optimal estimate
Jan 24, 2024
cda87ed
add Astar
Jan 24, 2024
871783c
deflat before estimate background and before fit chromatic psf
Jan 24, 2024
6cf4eb6
increase min bounds for Moffat gamma parameter from 0.1 to 0.5 pixel
Jan 24, 2024
7189b32
debug saving of parameters after FFM
Jan 24, 2024
cb4042f
debug saving of parameters after FFM
Jan 24, 2024
b24458f
implement flat and star field in ffm simulate
Jan 24, 2024
3bef660
add mock flat full of ones for CTIO
Jan 24, 2024
9456f10
add flat to image simulation
Jan 25, 2024
185afc0
add flat to full chain
Jan 25, 2024
ef1b2a7
cleaning
Jan 25, 2024
827fa4d
move Astar parameter position after B
Jan 25, 2024
7dc7191
remove B from SpectrogramModel and SpectrumModel: these classes compu…
Jan 25, 2024
161792f
add flat and starfield fits in fit_spectrogram
Jan 25, 2024
eba2234
debug save and load of flats and star fields
Jan 25, 2024
5676a68
cleaning and comments
Jan 26, 2024
90ef963
remove starfield before 1D and 2D extractions
Jan 26, 2024
67d261d
better starfield simulation: amplitude and ref position is set by mai…
Jan 26, 2024
f543de2
include flat and starfield in test_fullchain
Jan 28, 2024
ea07482
merge with master v3.0.3
Jan 28, 2024
e9bf965
repair make starfield
Jan 28, 2024
1e69134
repair make starfield
Jan 28, 2024
ea5019b
repiar test
Jan 28, 2024
71f25cb
introduce SPECTRACTOR_SIMULATE_STARFIELD parameter; introduce Image.s…
Jan 29, 2024
2817a1d
cleaning
Jan 29, 2024
e5a7b99
repair full chain test
Jan 29, 2024
f6dde98
if flat exists, flatten the image for the plot
jeremyneveu Feb 14, 2024
9f46399
correct log message
jeremyneveu Feb 14, 2024
ef4485b
typo in SPECTRACTOR_ATMOSPHERE_SIM
Feb 14, 2024
fcd058a
Merge branch 'v3.1.0' of github.com:LSSTDESC/Spectractor into v3.1.0
jeremyneveu Feb 14, 2024
81fa189
auxtel gain is 1.3
Feb 16, 2024
9a21aff
new __repr__ for FitParameters
Feb 16, 2024
b74926c
Merge branch 'v3.1.0' of github.com:LSSTDESC/Spectractor into v3.1.0
jeremyneveu Feb 16, 2024
2fe685c
debug rebinning of flat flield
jeremyneveu Feb 19, 2024
5d80062
add mask in Image class and spectractor extraction
Feb 19, 2024
dcb9d52
plot mask in debug plot
Feb 19, 2024
3652a21
crop spectrogram mask
Feb 19, 2024
6f85ee1
debug plot
Feb 19, 2024
08b4267
rebin mask
Feb 19, 2024
19e1f98
don't save None spectrogram mask
Feb 19, 2024
08a6f6b
debug rebinning of mask
Feb 19, 2024
e5d30b4
debug saving of None spectrogram_mask
Feb 19, 2024
e93e7e0
debug saving of None spectrogram_mask
Feb 19, 2024
fdbb5f7
debug saving of None spectrogram_mask
Feb 19, 2024
afcbd9b
better printing of parameters
Feb 20, 2024
56adb25
debug background box
Feb 20, 2024
c5005c4
don't crop spectrogram_data if already cropped
Feb 22, 2024
c17a971
fix Yc params for order 2 and 3 but Yc0; mask spectrogram region fit …
jeremyneveu Feb 23, 2024
31e960b
mask spectrogram region fit edges when not the maximum number of shee…
jeremyneveu Feb 23, 2024
29f738c
mask spectrogram region fit edges when not the maximum number of shee…
jeremyneveu Feb 23, 2024
ebc2a5f
Merge branch 'v3.1.0' of github.com:LSSTDESC/Spectractor into v3.1.0
jeremyneveu Feb 23, 2024
9a1e1e6
image simulation adapted for auxtel
Feb 23, 2024
2b62bbe
load_auxtel adapted for data and simulations
Feb 23, 2024
20a4b59
change REBIN key into CCD_REBIN key
Feb 23, 2024
9a6e861
imshow with no interpolation for better plot analysis
Feb 23, 2024
9d3b201
cleaning
Feb 23, 2024
34773e7
auxtel full chain simulation
Feb 23, 2024
539e601
auxtel full chain simulation
Feb 23, 2024
44e85cf
argh with ratio files
Feb 23, 2024
fbb3354
merge
Feb 23, 2024
3fbc609
ratio files with extrapolation from hologram transmission fit
Feb 23, 2024
3acb808
rename more explicitely get_boundaries into set_rectangular_boundarie…
Mar 6, 2024
3acf878
change arguments of get_sparse_indices to use boundaries (faster and …
Mar 6, 2024
3c187b8
image simulation using non squares images and rebinned spectra
Mar 6, 2024
9c8faf0
enlarge the next diffraction order footprints to fit inside main orde…
Mar 7, 2024
f1bafc8
possibility to choose between different parametrisation: legendre or …
Mar 7, 2024
f5caf63
possibility to choose between different parametrisation: legendre or …
Mar 7, 2024
bd2ee6b
possibility to choose between different parametrisation: legendre or …
Mar 7, 2024
f69284e
possibility to choose between different parametrisation: legendre or …
Mar 7, 2024
4368bd9
extrapolate ratios for holo4_003 in UV with hologram transmission model
jeremyneveu Mar 7, 2024
11d95a3
Merge branch 'v3.1.0' of github.com:LSSTDESC/Spectractor into v3.1.0
jeremyneveu Mar 7, 2024
7b1363b
enlarge boundaries for next diffraction order footprints, free shift_…
Mar 7, 2024
e70f5e1
set_bounds adapted to polynomial parametrisation
Mar 8, 2024
9600215
set_bounds adapted to polynomial parametrisation
Mar 8, 2024
98ce0fc
cancel A3
Mar 8, 2024
97d10e6
fix only y_c_0_1 and y_c_1_1
Mar 8, 2024
542175c
more robust way to save LBDAS_T and AMPLI_T
Mar 8, 2024
2730108
fix shift_y and angle but fit all y_c parameters (works better for fi…
Mar 8, 2024
8c8e4a7
allow for negative y_c values with bounds
Mar 8, 2024
fb6573c
initialize bounds for chromaticpsf1d
jeremyneveu Mar 8, 2024
afec03f
rescale abscissa x for polynoms event when they are canonical (better…
Mar 14, 2024
5bda1fb
fix wavelength evolution for PSF parameters of higher order diffracti…
Mar 14, 2024
0b14777
reorder paramaters to have all fixed psf parameters together
Mar 14, 2024
d926400
fit shift_x
Mar 14, 2024
56de744
fit A1, angle and shift_y first to prepare the fit
Mar 14, 2024
8ffb1d0
change unit of reso parameter in labels (nm)
Mar 14, 2024
d0f987b
more robust way to set weight of outliers to zero
Mar 14, 2024
b8423ea
restrict bounds for moffat parameters alpha=(1.1,10) and gamma > 1 pixel
Mar 14, 2024
c1459e9
fit temporarily A1 to help finding the trace of the spectrogram with …
Mar 14, 2024
27d1e51
set very high uncertainties below 380nm and above 1000nm to ease the …
jeremyneveu Mar 14, 2024
9aa2545
SDSS g filter
jeremyneveu Mar 14, 2024
ff1bd72
Merge branch 'v3.1.0' of github.com:LSSTDESC/Spectractor into v3.1.0
jeremyneveu Mar 14, 2024
99becf5
debug position of B, Astar
Mar 14, 2024
43b1a7f
sketch of auxtel full chain test
Mar 14, 2024
956d835
Merge branch 'v3.1.0' of github.com:LSSTDESC/Spectractor into v3.1.0
jeremyneveu Mar 14, 2024
138e880
keep tentative to set bounds for polynomial parameters
Mar 15, 2024
f89d3a2
for CTIO PSF_POLY_ORDER=2
Mar 15, 2024
5d4de72
Merge branch 'v3.1.0' of github.com:LSSTDESC/Spectractor into v3.1.0
jeremyneveu Mar 15, 2024
9ebe370
PSF_POLY_ORDER=2
Apr 9, 2024
e0a7051
repair runAstrometry.py
Apr 9, 2024
8d1f441
output_filename as argument of ImageSim
Apr 9, 2024
58cce5a
stardice load image with detrend and load WCS if there
Apr 9, 2024
e4e0f63
add stardice full chain test
Apr 9, 2024
185fdce
red circle for target position in plot_image_simple
Apr 9, 2024
9da2cec
TODO: add good pressure temp and humiduty values if available in header
Apr 9, 2024
3ee9a7a
merge with v3.1.0
Apr 9, 2024
e9de2e0
merge with v3.1.0
Apr 9, 2024
42ee3ac
add good pressure temp and humidity values if available in header
Apr 10, 2024
be71913
psf_fit_reg_param = 1
Apr 15, 2024
efa12f7
SPECTRACTOR_SIMULATE_STARFIELD = False
Apr 15, 2024
5c5f03d
plot spectrum at the end of psf1d
Apr 15, 2024
4621f97
save trials to improve stardice reduction months after I have written…
Aug 7, 2024
085700d
include masked data in get_bad_indices; better use of self.mask; rese…
Sep 5, 2024
a61ef53
use scipy.simpsons
Sep 5, 2024
ce2e7af
add mask in fit_transverse_PSF1D_profile
Sep 5, 2024
7341b46
add mask in fit_transverse_PSF1D_profile
Sep 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions config/auxtel.ini
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@ SPECTRACTOR_FIT_TIMEOUT_PER_ITER = 1200
# maximum time per gradient descent before TimeoutError in seconds
SPECTRACTOR_FIT_TIMEOUT = 7200
# library to compute atmospheric transmission: none, libradtran, getobsatmo
SPECTRACTOR_ATMOSPHERE_SIM = getobstamo
SPECTRACTOR_ATMOSPHERE_SIM = getobsatmo
# simulate star field with Gaia catalog: False, True
SPECTRACTOR_SIMULATE_STARFIELD = False

[instrument]
# instrument name
Expand Down Expand Up @@ -55,7 +57,7 @@ CCD_PIXEL2ARCSEC = 0.0952
# approximate maximum ADU output of the CCD
CCD_MAXADU = 170000
# electronic gain : elec/ADU
CCD_GAIN = 1.1
CCD_GAIN = 1.3
# rebinning of the image in pixel
CCD_REBIN = 2

Expand Down
4 changes: 3 additions & 1 deletion config/ctio.ini
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ SPECTRACTOR_DECONVOLUTION_PSF2D = True
SPECTRACTOR_DECONVOLUTION_FFM = True
# library to compute atmospheric transmission: none, libradtran, getobsatmo
SPECTRACTOR_ATMOSPHERE_SIM = none
# simulate star field with Gaia catalog: False, True
SPECTRACTOR_SIMULATE_STARFIELD = True

[instrument]
# instrument name
Expand Down Expand Up @@ -102,7 +104,7 @@ PIXWIDTH_BOXSIZE = 40
# the PSF model: Gauss, Moffat or MoffatGauss
PSF_TYPE = Moffat
# the order of the polynomials to model wavelength dependence of the PSF shape parameters
PSF_POLY_ORDER = 4
PSF_POLY_ORDER = 2
# regularisation parameter for the chisq minimisation to extract the spectrum
PSF_FIT_REG_PARAM = 0.04
# step size in pixels for the first transverse PSF1D fit
Expand Down
4 changes: 4 additions & 0 deletions config/default.ini
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ SPECTRACTOR_FIT_TIMEOUT_PER_ITER = 600
SPECTRACTOR_FIT_TIMEOUT = 3600
# library to compute atmospheric transmission: none, libradtran, getobsatmo
SPECTRACTOR_ATMOSPHERE_SIM = none
# simulate star field with Gaia catalog: False, True
SPECTRACTOR_SIMULATE_STARFIELD = False

[instrument]
# instrument name
Expand Down Expand Up @@ -119,6 +121,8 @@ PSF_TYPE = Moffat
PSF_POLY_ORDER = 2
# regularisation parameter for the chisq minimisation to extract the spectrum
PSF_FIT_REG_PARAM = 0.01
# polynomial type: must be polynomial or legendre
PSF_POLY_TYPE = "polynomial"

[detection line algorithm parameters]
# order of the background polynome to fit
Expand Down
12 changes: 7 additions & 5 deletions config/stardice.ini
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ THROUGHPUT_DIR = simulation/StarDiceThroughput/

[pipeline]
# method to get target centroid, choose among: guess, fit, WCS
SPECTRACTOR_FIT_TARGET_CENTROID = WCS
SPECTRACTOR_FIT_TARGET_CENTROID = fit
# method to get image rotation angle: False, disperser, hessian
SPECTRACTOR_COMPUTE_ROTATION_ANGLE = disperser
# parameter to prevent from background subtraction
Expand All @@ -19,6 +19,8 @@ SPECTRACTOR_DECONVOLUTION_FFM = True
SPECTRACTOR_DECONVOLUTION_SIGMA_CLIP = 100
# library to compute atmospheric transmission: none, libradtran, getobsatmo
SPECTRACTOR_ATMOSPHERE_SIM = getobsatmo
# simulate star field with Gaia catalog: False, True
SPECTRACTOR_SIMULATE_STARFIELD = False

[instrument]
# instrument name
Expand Down Expand Up @@ -88,15 +90,15 @@ ROT_ANGLE_MAX = 30

[spectrum range]
# minimum wavelength for spectrum extraction (in nm)
LAMBDA_MIN = 350
LAMBDA_MIN = 330
# maximum wavelength for spectrum extraction (in nm)
LAMBDA_MAX = 1000
# spectrum order to extract
SPEC_ORDER = 1

[background subtraction parameters]
# half transverse width of the signal rectangular window in pixels
PIXWIDTH_SIGNAL = 10
PIXWIDTH_SIGNAL = 20
# distance from dispersion axis to analyse the background in pixels
PIXDIST_BACKGROUND = 20
# transverse width of the background rectangular window in pixels
Expand All @@ -108,9 +110,9 @@ PIXWIDTH_BOXSIZE = 40
# the PSF model: Gauss, Moffat or MoffatGauss
PSF_TYPE = Moffat
# the order of the polynomials to model wavelength dependence of the PSF shape parameters
PSF_POLY_ORDER = 4
PSF_POLY_ORDER = 2
# regularisation parameter for the chisq minimisation to extract the spectrum
PSF_FIT_REG_PARAM = 0.1
PSF_FIT_REG_PARAM = 1
# step size in pixels for the first transverse PSF1D fit
PSF_PIXEL_STEP_TRANSVERSE_FIT = 1
# PSF is not evaluated outside a region larger than max(PIXWIDTH_SIGNAL, PSF_FWHM_CLIP*fwhm) pixels
Expand Down
6 changes: 3 additions & 3 deletions runAstrometry.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
from spectractor.astrometry import Astrometry
from spectractor.logbook import LogBook
from spectractor.config import load_config, apply_rebinning_to_parameters
from spectractor.extractor.images import Image


if __name__ == "__main__":
Expand Down Expand Up @@ -50,7 +51,6 @@

logbook = LogBook(logbook=args.logbook)
for file_name in file_names:
disperser_label = args.disperser_label
if parameters.OBS_NAME == "CTIO":
tag = file_name.split('/')[-1]
tag = tag.replace('sim_', 'reduc_')
Expand All @@ -66,8 +66,8 @@
ypos = float(ypos)
guess = [xpos, ypos]
target_label = args.target_label
a = Astrometry(file_name, target_label=target_label, disperser_label=disperser_label,
output_directory=args.output_directory)
image = Image(file_name, config=args.config)
a = Astrometry(image, output_directory=args.output_directory)
extent = ((int(max(0, xpos - radius)), int(min(xpos + radius, parameters.CCD_IMSIZE))),
(int(max(0, ypos - radius)), int(min(ypos + radius, parameters.CCD_IMSIZE))))
gaia_min_residuals = a.run_full_astrometry(extent=extent, maxiter=int(args.maxiter))
Expand Down
2 changes: 1 addition & 1 deletion runImageSim.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,4 +49,4 @@
continue

image = ImageSim(file_name, spectrum_file_name, args.output_directory, A2=1,
psf_poly_params=psf_poly_params, with_stars=False)
psf_poly_params=psf_poly_params, with_starfield=False)
2 changes: 1 addition & 1 deletion runSimulator.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,6 @@
atmgrid = AtmosphereGrid(file_name)
image = ImageSim(file_name, spectrum_file_name, args.output_directory, A1=1, A2=1,
pwv=5, ozone=300, aerosols=0.03,
psf_poly_params=None, with_stars=True)
psf_poly_params=None, with_starfield=True)
sim_file_name = args.output_directory + tag.replace('reduc_', 'sim_')
Spectractor(sim_file_name, args.output_directory, target, [xpos, ypos], disperser_label, args.config)
2 changes: 1 addition & 1 deletion spectractor/_version.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,5 +28,5 @@
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

__version__ = '3.0.3'
__version__ = '3.1.0'
__version_info__ = tuple(map(int, __version__.split('.')))
4 changes: 4 additions & 0 deletions spectractor/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,10 @@ def load_config(config_filename, rebin=True):
if parameters.PIXWIDTH_BOXSIZE > parameters.PIXWIDTH_BACKGROUND:
raise ValueError(f'parameters.PIXWIDTH_BOXSIZE must be smaller than parameters.PIXWIDTH_BACKGROUND (or equal).')

# check consistency
if parameters.PSF_POLY_TYPE not in ["polynomial", "legendre"]:
raise ValueError(f'parameters.PSF_POLY_TYPE must be either "polynomial" or "legendre". Got {parameters.PSF_POLY_TYPE=}')

# check presence of atmospheric simulation packages
if parameters.SPECTRACTOR_ATMOSPHERE_SIM.lower() not in ["none", "libradtran", "getobsatmo"]:
raise ValueError(f'parameters.SPECTRACTOR_ATMOSPHERE_SIM must be either ["none", "libradtran", "getobsatmo"]. '
Expand Down
4 changes: 2 additions & 2 deletions spectractor/extractor/background.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ def make_source_mask(data, nsigma, npixels, mask=None, sigclip_sigma=3.0,

Parameters
----------
data : 2D `~numpy.ndarray`
data : np.ndarray
The 2D array of the image.
nsigma : float
The number of standard deviations per pixel above the ``background``
Expand Down Expand Up @@ -236,7 +236,7 @@ def extract_spectrogram_background_sextractor(data, err, ws=(20, 30), mask_signa
>>> from spectractor import parameters
>>> parameters.DEBUG = True
>>> psf = MoffatGauss()
>>> s0 = ChromaticPSF(psf, Nx=100, Ny=200, saturation=1000)
>>> s0 = ChromaticPSF(psf, Nx=100, Ny=300, saturation=1000)
>>> params = s0.generate_test_poly_params()
>>> saturation = params[-1]
>>> data = s0.evaluate(s0.set_pixels(mode="1D"), poly_params=params)
Expand Down
Loading
Loading