PyFstat issueshttps://gitlab.aei.uni-hannover.de/GregAshton/PyFstat/-/issues2020-01-14T10:51:26Zhttps://gitlab.aei.uni-hannover.de/GregAshton/PyFstat/-/issues/8gridsearch output format and (t0,tau)band use case2020-01-14T10:51:26ZDavid Keiteldavid.keitel@ligo.orggridsearch output format and (t0,tau)band use caseFor the implementation of explicit t0,tau search bands in !7 I'm a bit in conflict with PyFstat's output file format from gridded searches, which so far is a simple headerless file with columns
`startTime endTime F0 F1 F2 Alpha Delta 2F`...For the implementation of explicit t0,tau search bands in !7 I'm a bit in conflict with PyFstat's output file format from gridded searches, which so far is a simple headerless file with columns
`startTime endTime F0 F1 F2 Alpha Delta 2F`.
First a question: the original transient search style you had in mind was probably to pass arrays of minStartTime, maxStartTime and then the search is an (up to) 7D grid of ComputeFStat calls over all those columns? That's how I parse GridSearch.get_input_data_array too.
Whereas the !7 version is closer to CFSv2, max-5D search and transient postprocessing over a t0,tau grid at each point. I can see value in both approaches and we should be able to support both in the same interface and output format.
In the !7 implementation, it now still writes out the same overall minStartTime and maxStartTime into the first 2 columns for every Doppler candidate, and in the last column max(2F) maximised over the t0,tau grid. Problem with that is information loss, as a symptom of which running a new GridSearch with added transientWindowType, t0Band, tauBand arguments in a directory with an existing output file from identical Doppler parameters will not trip check_old_data_is_okay_to_use and hence not actually rerun the search.
Simple solutions:
1. in the (t0Band,tauBand) usecase, replace the first 2 columns by the ML estimators (t0,t0+tau at which F=maxF)
1. in the (t0Band,tauBand) usecase, add extra columns
1. generally add a header with all search parameters (also beneficial for data archiving and external postprocessing)
More complicated solution:
1. wrap lalpulsar file-writing functions
What do you think @GregAshton ?David Keiteldavid.keitel@ligo.orgDavid Keiteldavid.keitel@ligo.orghttps://gitlab.aei.uni-hannover.de/GregAshton/PyFstat/-/issues/10reminder: add header to output files2020-02-05T10:47:15ZDavid Keiteldavid.keitel@ligo.orgreminder: add header to output filesLow-priority reminder issue for text file headers, as briefly discussed in #8:
> David: generally add a header with all search parameters (also beneficial for data archiving and external postprocessing
> Greg: Nevertheless, adding a ...Low-priority reminder issue for text file headers, as briefly discussed in #8:
> David: generally add a header with all search parameters (also beneficial for data archiving and external postprocessing
> Greg: Nevertheless, adding a header as is done by CFSv2 is something on my to-do list (also for the MCMC output)David Keiteldavid.keitel@ligo.orgDavid Keiteldavid.keitel@ligo.orghttps://gitlab.aei.uni-hannover.de/GregAshton/PyFstat/-/issues/11Move transient gridded search to separate class2020-01-14T10:51:35ZGregory AshtonMove transient gridded search to separate classHi @dkeitel ,
Currently, I think you are using `GridSearch` to run your transient grid searches, is that correct? If you continue to develop this, it may make sense to create a specific subclass, say `TransientGridSearch`. Basically, yo...Hi @dkeitel ,
Currently, I think you are using `GridSearch` to run your transient grid searches, is that correct? If you continue to develop this, it may make sense to create a specific subclass, say `TransientGridSearch`. Basically, you do something like
```
class TransientGridSearch(GridSearch)
def __init__ ...
```
This way, `TransientGridSearch` inherits all methods from `GridSearch`, but then overwrite any parts it needs to modify.
The benefit is, any other methods which subclass `GridSearch`, doesn't need to worry about the transient parts (and hence changes to the transient parts won't break anything else).
Is there any reason why this wouldn't work? If not, I'll have a look at creating a patch and seeing what you thinkGregory AshtonGregory Ashtonhttps://gitlab.aei.uni-hannover.de/GregAshton/PyFstat/-/issues/22teach TransientGridSearch to re-use atoms and Fmn from disk2020-02-05T10:56:57ZDavid Keiteldavid.keitel@ligo.orgteach TransientGridSearch to re-use atoms and Fmn from disk`GridSearch` already has logic to re-use results from disk, if inputs match. `TransientGridSearch` inherits this but isn't able to do the same for the actual transient-relevant outputs, just the standard persistent-2F grid file.
Best wa...`GridSearch` already has logic to re-use results from disk, if inputs match. `TransientGridSearch` inherits this but isn't able to do the same for the actual transient-relevant outputs, just the standard persistent-2F grid file.
Best way to do this would probably be through overriding the `check_old_data_is_okay_to_use()` method.David Keiteldavid.keitel@ligo.orgDavid Keiteldavid.keitel@ligo.org