PyFstat issueshttps://gitlab.aei.uni-hannover.de/GregAshton/PyFstat/-/issues2020-01-14T10:51:35Zhttps://gitlab.aei.uni-hannover.de/GregAshton/PyFstat/-/issues/11Move transient gridded search to separate class2020-01-14T10:51:35ZGregory AshtonMove transient gridded search to separate classHi @dkeitel ,
Currently, I think you are using `GridSearch` to run your transient grid searches, is that correct? If you continue to develop this, it may make sense to create a specific subclass, say `TransientGridSearch`. Basically, yo...Hi @dkeitel ,
Currently, I think you are using `GridSearch` to run your transient grid searches, is that correct? If you continue to develop this, it may make sense to create a specific subclass, say `TransientGridSearch`. Basically, you do something like
```
class TransientGridSearch(GridSearch)
def __init__ ...
```
This way, `TransientGridSearch` inherits all methods from `GridSearch`, but then overwrite any parts it needs to modify.
The benefit is, any other methods which subclass `GridSearch`, doesn't need to worry about the transient parts (and hence changes to the transient parts won't break anything else).
Is there any reason why this wouldn't work? If not, I'll have a look at creating a patch and seeing what you thinkGregory AshtonGregory Ashtonhttps://gitlab.aei.uni-hannover.de/GregAshton/PyFstat/-/issues/10reminder: add header to output files2020-02-05T10:47:15ZDavid Keiteldavid.keitel@ligo.orgreminder: add header to output filesLow-priority reminder issue for text file headers, as briefly discussed in #8:
> David: generally add a header with all search parameters (also beneficial for data archiving and external postprocessing
> Greg: Nevertheless, adding a ...Low-priority reminder issue for text file headers, as briefly discussed in #8:
> David: generally add a header with all search parameters (also beneficial for data archiving and external postprocessing
> Greg: Nevertheless, adding a header as is done by CFSv2 is something on my to-do list (also for the MCMC output)David Keiteldavid.keitel@ligo.orgDavid Keiteldavid.keitel@ligo.orghttps://gitlab.aei.uni-hannover.de/GregAshton/PyFstat/-/issues/8gridsearch output format and (t0,tau)band use case2020-01-14T10:51:26ZDavid Keiteldavid.keitel@ligo.orggridsearch output format and (t0,tau)band use caseFor the implementation of explicit t0,tau search bands in !7 I'm a bit in conflict with PyFstat's output file format from gridded searches, which so far is a simple headerless file with columns
`startTime endTime F0 F1 F2 Alpha Delta 2F`...For the implementation of explicit t0,tau search bands in !7 I'm a bit in conflict with PyFstat's output file format from gridded searches, which so far is a simple headerless file with columns
`startTime endTime F0 F1 F2 Alpha Delta 2F`.
First a question: the original transient search style you had in mind was probably to pass arrays of minStartTime, maxStartTime and then the search is an (up to) 7D grid of ComputeFStat calls over all those columns? That's how I parse GridSearch.get_input_data_array too.
Whereas the !7 version is closer to CFSv2, max-5D search and transient postprocessing over a t0,tau grid at each point. I can see value in both approaches and we should be able to support both in the same interface and output format.
In the !7 implementation, it now still writes out the same overall minStartTime and maxStartTime into the first 2 columns for every Doppler candidate, and in the last column max(2F) maximised over the t0,tau grid. Problem with that is information loss, as a symptom of which running a new GridSearch with added transientWindowType, t0Band, tauBand arguments in a directory with an existing output file from identical Doppler parameters will not trip check_old_data_is_okay_to_use and hence not actually rerun the search.
Simple solutions:
1. in the (t0Band,tauBand) usecase, replace the first 2 columns by the ML estimators (t0,t0+tau at which F=maxF)
1. in the (t0Band,tauBand) usecase, add extra columns
1. generally add a header with all search parameters (also beneficial for data archiving and external postprocessing)
More complicated solution:
1. wrap lalpulsar file-writing functions
What do you think @GregAshton ?David Keiteldavid.keitel@ligo.orgDavid Keiteldavid.keitel@ligo.orghttps://gitlab.aei.uni-hannover.de/GregAshton/PyFstat/-/issues/7add switches to override check_old_data_is_okay_to_use?2020-01-14T10:50:41ZDavid Keiteldavid.keitel@ligo.orgadd switches to override check_old_data_is_okay_to_use?Both the grid and MCMC search classes have check_old_data_is_okay_to_use methods, which are a good idea in principle, but might need to get quite a bit more involved to check all possible differences between search calls in all use cases...Both the grid and MCMC search classes have check_old_data_is_okay_to_use methods, which are a good idea in principle, but might need to get quite a bit more involved to check all possible differences between search calls in all use cases. (E.g. the simple grid output format currently cannot be used to distinguish between a search with and without t0,tau bands, which I'll open another issue about.)
So a nice convenience feature, at least for use during development, would be a 'reusedata' argument, default True, but when set to False just skips that check. If you like it, I can make a patch.Gregory AshtonGregory Ashton