-
Notifications
You must be signed in to change notification settings - Fork 6
Expand file tree
/
Copy pathReleaseNotes
More file actions
1128 lines (1090 loc) · 58.2 KB
/
Copy pathReleaseNotes
File metadata and controls
1128 lines (1090 loc) · 58.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
# -*- mode: org -*-
* Version 0.6.4 - 2014-11-24
** Requirements
The only requirement is a python-installation. The main testing and
development has been done with 2.7 and 2.6. Python 2.5 should work
but not for all utilities. Currently 2.6, 2.7 and 3.4 are used
during development.
Certain functionalities need special Python-libraries. The main one
is =numpy= for plotting. Other libraries that might be of interest
are =matplotlib=, =pandas= and =IPython=. For
=pyFoamDisplayBlockmesh.py= the libraries =PyQt4= and =vtk= are
needed. Other libraries are tested for and reported by
=pyFoamVersion.py= but not required (only install if you're sure
that you need it)
** Future changes
*** Redundant utilities =pyFoamJoinCSV.py= and =pyFoamConvertToCSV.py= unified
These two utilities are almost indistinguishable and will be
unified into one
** Major changes
*** Multi-line regular expressions in =customRegexp=
If in =customRegexp= an =expr= is found with =\n= then this
expression is matched over multiple consecutive lines. Types like
=dynamic= work as usual.
This makes it possible to match for instance the output of the
=forces=-function objects
*** Enhancement of =pyFoamPrepare.py=
The utility which was introduced in the last version is becomong
more usable and will be central to all things that set up the case
(for instance a special =ClusterJob=)
*** Enhancements of the CSV-utilities
These utilities are now more flexible and allow writing and
reading of Excel-files too
*** Environment variable =PYFOAM_SITE_DIR= and =PYFOAM_DIR=
Both variables are not necessary but =PYFOAM_SITE_DIR= allows
consistent insertion of site-specific libraries and utilities.
=PYFOAM_DIR= is set by some Foam-distributions and tested for by
=pyFoamVersion.py=. It is supposed to point to the currents
PyFoam-installation.
=PYFOAM_SITE_DIR= points to a directory with site-specific scripts
and configurations. The sub-directories supported (and thested py
=pyFoamVersion.py=) are
- bin :: directory with scripts. It has to be added to the =PATH=
outside of PyFoam (for instance in the =.bashrc=)
- etc :: Optional directory which is searched for configuration
files. Priority is below the user settings and above the
global settings
- lib :: Optional directory to allow mixing in site-specific
library files. Imported as =PyFoam.Site=: For instance if
a file =Foo.py= is present in =lib= it can be imported as
=PyFoam.Site.Foo=. This directory does not have to be (in
fact: it *shouldn't*) added to =PYTHONPATH=
Purpose of =PYFOAM_SITE_DIR= is to allow administrators to provide
site-wide scripts and settings for all users on a site
** Incompatibilities
*** Option =--silent= removed from =pyFoamPrepareCase.py=
Option has been renamed to =--no-complain=
*** Keys in =RunDatabase= with column-names that contain upper-case letters change
SQLite does not support case-sensitive column-names (=s_max= and
=S_max= are the same). To change this the upper case letters in
the column names are replaced by an underscore and the letter
(=S_max= becomes =_s__max=)
This means that old databases might not be read correctly
*** Change in unique variable names in =pyFoamConvertToCSV.py=
The algorithm to make variable names unique has changed (basically
it uses the part of the filenames that differ) and scripts relying
on these names might fail
*** =PyFoam.IPython=-module renamed to =PyFoam.IPythonHelpers=
The name of the module crashed in certain instances (especially
unit-testing) with the regular =IPython=-library. To avoid these
crashes it has been renamed to =IPythonHelpers=. This raises two
potential problems:
- scripts that =import= the module have to be adapted to the new name
- IPython-notebooks created with =pyFoamIPythonNotebook.py= have
two imports pointing to this module. These notebooks have to be
adapted to be usable again
** Bugfixes
*** Templates in =pyFoamPrepareCase.py= did not keep permissions
This was a problem for script-templates which were not executable
any more. Fixed
*** =pyFoamComparator.py= failed due to circular dependency
This has been fixed by adding an import in =BasicRunner.py=
*** =pyFoamDumpRunDatabaseToCSV.py= fails if Pandas-data is requested
This is now fixed
*** =sort= for list broke code on Python 3
Some calls for =sort= still used the =cmp=-parameter which does
not exist for Python3 anymore. These calls have been replaced with
=key= and =reverse=
*** Changing the OF-version does not work in Python 3
Because the output of =subprocess= is now /binary/ instead of a
regular string. Fixed
*** =addData= in =PyFoamDataFrame= extrapolates for invalid values
This was due to incorrect use of the =interpolate=-method
*** =--keep-last= did not work for =pyFoamClearCase.py= and parallel cases
This was because there was a problem in the library code and the
utility did not consider the parallel time-steps. Fixed
*** =pyFoamDumpRunDatabaseToCSV.py= does not add basic run information
Basic run information was not added to the file. Now it is with
the prefix =runInfo//=
*** Restore of =FileBasisBackup= did not work
The logic for checking whether a file was "backupable" was
wrong. This affected the proper restore of files with utilities
for instance for =--write-all=
*** Remove circular dependency in =DataStructures=
According to the bug
http://sourceforge.net/p/openfoam-extend/ticketspyfoam/219/ it was
not possible to import =DataStructures= because of a circular
dependency with =FoamFileGenerator=. Fixed by moving an import to
the back of the file
** New features/Utilities
*** =pyFoamRunParameterVariation.py=
This utility takes a template case and a file specifying the
parameter variation and creates cases with the
=pyFoamPrepareCase.py=-engine, runs a solver on these cases and
collects the data into a database. The database can then be
extracted with =pyFoamDumpRunDatabaseToCSV.py=
*** =pyFoamBinarySize.py=
Calculates the size of the binaries in an OpenFOAM-installation
separated by compile-option
*** =pyFoamBlockMeshRewrite.py=
Assists the user in rewriting the =blockMeshDict= by doing simple,
but error-prone transformations. Assumes "sensible" formatting:
one block/vertex etc per line.
Sub-commands are:
- refine :: refines mesh by multiplying cell numbers in the blocks
- number :: Adds comments with the vertex numbers. Should help the
user when editing/modifying the mesh
- stripNumber :: Remove the comments added by =number=
- mergeVertices :: Adds vertices from other blockMeshes that
are not present in the current blockMesh
- renumberVertices :: Take another =blockMeshDict=, copy over the
=vertices=-section of that mesh and rewrite =blocks= and
=patches= so that they conform to these =vertices=. The
original =vertices= have to be a sub-set of the =vertices= in
the other mesh
- normalizePatches :: Rotates patches so that the lowest number is
in front
** Enhancements to Utilities
*** =pyFoamChangeBoundaryType.py= allows setting additional values
The option =--additional-values= allows specifying a dictionary
with additional values for the boundary (stuff that is needed by
=mappedWall= etc)
*** =pyFoamPrepareCase.py= now has OF-version and fork as defined variables
This allows to write case-templates that can distinguish between
different OF-versions
*** =pyFoamPrepareCase.py= now allows "overloading" another directory
Before doing anything else the contents of different directories
are copied into the current case. This allows for instance to use
tutorial cases as the basis for a case
*** =pyFoamIPythonNotebook.py= adds improvements to the notebook
Additional code added to the generated notebook:
- Code to change the default size of the plots
- Distribution-directories in subdirectories =distributions=
(generated by some =swak=-function objects) added
*** =pyFoamListCases.py= more tolerant to faulty =controlDict=
If the =controlDict= is acceptable to OpenFOAM but syntactically
incorrect for PyFoam (for instance because of a missing semicolon)
the utility does not fail anymore (but no data is collected for
that case).
*** =pyFoamDumpConfiguration.py= prints sections and keys alphabetically
This should make it easier to find items
*** =pyFoamJoinCSV.py= and =pyFoamConvertToCSV.py= read and write Excel-files
Both utilities now allow writing Excel-files
In addition to regular text files the first sheet from =xls=-files
can be read
*** Flexible variable filtering in =pyFoamJoinCSV.py= and =pyFoamConvertToCSV.py=
Now it is possible to filter for regular expressions
The functionality of the two utilities now is very similar and it
is possible that one of them will be discontinued
*** Columns in =pyFoamJoinCSV.py= and =pyFoamConvertToCSV.py= can be recalculated
The two utilities now can add columns or recalculate columns
based on the existing column values
*** Testing for =Numeric= removed from =pyFoamVersion.py=
Testing for the library =Numeric= library removed as it is no
longer supported as a fallback for =numpy=. Test also removed from
=setup.py=
** Enhancements to the Library
*** Subclass of =ClusterJob= that support =PrepareCase=
The class =PrepareCaseJob= supports cases that are set up with
=pyFoamPrepareCase.py=. Additional parameters to the constructor are
- the name of the parameter-file
- a list with the parameters. The list is composed of
name/value-pairs
*** Subclass of =ClusterJob= that support =RunParameterVariation=
The class =VariationCaseJob= supports cases that are set up with
=pyFoamRunParameterVariation.py=. Additional parameters to the constructor are
- the name of the parameter-file
- the name of the variations-file
*** =execute= in =PyFoam/Utilities= fails if script is not executable
The function checks if the file exists and is *not*
executable. The program fails in that case
*** =foamVersion= uses a separate wrapper class for =tuple=
This ensures that it is printed in a form that is valid in
OF-dictionaries
*** Move calculation of disk usage to =Utilities=
This has until now only been used in =ListCases= but moved to a
separate method/function =diskUsage= in the =Utilities=-module
*** Enhancement of =--help=
Added the possibility to have an epilog and usage examples with
the =epilog= and =examples=-keyword arguments for applications.
These and descriptions now have the possibility for line-breaks:
if two line-breaks are encountered in the text a new paragraph is
created
*** =which=-routine in =Utitlities= uses native Python-routine
For Python-version where =shutil= has a =which=-function this is
used instead of calling an external program
*** =FileBasis= now allows file handles instead of the filename
This currently only works for reading, Backups, zipping etc won't
work but it makes algorithms more flexible
*** =BlockMesh= doesn't force writing to file anymore
Instead content is stored in memory. Old behaviour is the default
to preserve compatibility with old scripts
*** Additional methods for =BlockMesh=-class
- numberVertices :: Adds comments with the vertex numbers to the
vertices
*** =LineReader= allows keeping spaces on left
Previous behaviour was stripping all spaces from the lines. Now
the left hand spaces can be ket. Old behaviour is still default
for compatibility
*** =TemplateFile= now allows writing of assignment-results in file
This allows faster debugging of template-files. This can be
enabled with a switch in the utilities using templates
*** =SolverJob= now allows passing of parameters to the solver
And additional parameter =solverArgs= will now be passed to the
solver (if the solver accepts arguments)
*** =SpreadsheetData= now allows reading from an Excel file
During construction if an Excel-file is specified and the
=xlrd=-library and =pandas= are installed then the first sheet in
the file is read
*** =SpreadsheetData= allows recalculating columns
Columns can be recalculated using expressions. This includes other
data items. Currently present column names are available as
variables. There is also a variable =data= that can be subscripted
for items that are not valid variable names. A variable =this=
points to the item to be recalculated
** Known bugs
*** Timelines not forgotten for multiple runner calls
This manifests with =pyFoamRunParameterVariation.py=. The custom
timelines are still kept in memory. Not a problem. Just annoying
* Version 0.6.3 - 2014-06-23
** Requirements
The only requirement is a python-installation. The main testing and
development has been done with 2.7 and 2.6. Python 2.5 should work
but not for all utilities. Unit tests run on Python 3.4 but it is
currently not used in a production environment (reports of success
or failure most welcome)
Certain functionalities need special Python-libraries. The main one
is =numpy= for plotting. Other libraries that might be of interest
are =matplotlib=, =pandas= and =IPython=. For
=pyFoamDisplayBlockmesh.py= the libraries =PyQt4= and =vtk= are
needed. Other libraries are tested for and reported by
=pyFoamVersion.py= but not required (only install if you're sure
that you need it)
** Major changes
*** Version changing supports forks of OpenFOAM
Now =pyFoam= supports different versions of OpenFOAM for switching.
Out of the box =openfoam= and =extend= are supported. If only the
version number is specified (for instance by =--foamVersion=1.7.x=)
and such a version exists only for one fork it is correctly
expanded with the correct fork( in the example with
=openfoam-1.7.x=). If more than one fork has the same version then
the fork name has to be specified as well
Note: additional forks can be easily specified with the
configurations. In section =OpenFOAM= the parameter =forks= has to
be extended. For each new fork a =dirpatterns= and
=installation=-parameter has to be specified
** Incompatibilities
*** Change of command interface of =pyFoamSTLUtility.py=
The selection of what is to be done is now selected by subcommands
instead of options. This will break scripts using this
*** If =0.org= is present =pyFoamCloneCase.py= and =pyFoamPackCase.py= ignore =0=
The reason is that the utilities assume that this directory is
produced from =0.org=
** Bugfixes
*** PlotWatcher has long times between updates if pickling takes long
The reason was that it used the same throttling that made sense
for the PlotRunner. Fixed
*** =pyFoamPVSnapshot.py= fails for newer paraview-versions
Reason is that the class =vtkPythonStdStreamCaptureHelper= does
not support =isatty=
*** SamplePlot failed when valueNames are unspecified
Reported in
https://sourceforge.net/apps/mantisbt/openfoam-extend/view.php?id=208
and fixed
*** =pyFoamTimelinePlot.py= failed Numpy/Pandas output of vector fields
Vector fields only were added to the data fields if they were the
first in the list. Fixed
*** =alternateAxis= ignored for slave
This is now fixed. The alternate values have to be specified in
the master (specifying in the slave gives an error)
*** =pyFoamCaseReport.py= more stable for binary =boundary=-files
Usually these files are =ascii= (even if the header says
=binary=). In some cases the parsing failed for these. Fixed by
enforcing reading as =ascii=. Can be switched off
*** =SpreadsheetData= returns data which breaks certain Pandas-operations
The reason was that if there were duplicate times in the table the
index was non-unique which certain Pandas-operations don't
appreciate. Solved by dropping duplicate times. Can be switched off
*** =pyFoamCloneCase.py= added duplicates to the archive
If things are specified twice they were added twice. Now it is
checked whether the item already exists in the tar-file before
adding them
*** =nonuniform= of length 3 not correctly printed
The reason was that this was interpreted as a vector and the
numeric prefix was removed. Reported at
http://sourceforge.net/apps/mantisbt/openfoam-extend/view.php?id=218
Fixed by introducing an extra parameter to =FoamFileGenerator=
** New features/Utilities
*** =pyFoamPrepareCase.py= for case preparation
This utility aims to reduce the need for boilerplate scripts to
set up cases. The steps it executes are
1. Clear old data from the case (including processor directories)
2. if a folder =0.org= is present remove the =0= folder too
3. go through all folders and for every found file with the
extension =.template= do template expansion using the
pyratemp-engine
4. create a mesh (either by using a script or if a =blockMeshDict=
is present by running blockMesh. If none of these is present
assume that there is a valid mesh present)
5. copy every =foo.org= that is found to to =foo= (recursively if directory)
6. do template replacement for every =.postTemplate=
7. execute another preparation script
*** =pyFoamIPythonNotebook.py= for generating and manipulating IPython-notebooks
This utility creates and manipulates IPython-notebooks for
analyzing OpenFOAM cases. It has a number of subcommands:
- create :: this is the main command. All other commands assume
that the notebooks they work with were created with
this.
The utility looks at the case specified and creates a
notebook that has the capabilities to quickly build a
report about the case:
- reporting general properties of the case. This
basically is the capability of the
=pyFoamCaseReport.py=-utility
- It searches for data that can be visualized by
=pyFoamTimelinePlot.py= or =pyFoamSamplePlot.py= and
generates selectors that allow the user to select
which data to import. The selectors import the data as
Pandas-=DataFrames= and create the commands
necessary to do this. It is recommended to erase the
selector afterwards
- Selectors for pickled case data and pickled plot
generated by PyFoam
- Capability to store read data *in* the notebook
file. This feature is experimental and has
performance issues for medium to large datasets
The created notebook can be executed but needs to be
edited to be useful
- clear :: removes selected cells (but only cells created with
=create=) or output from the notebook.
- info :: prints information about the notebook
- copy :: copies notebook to a different case and rewrites it so
that data is read from that case
Recommended way of working with this utility is
1. Create notebook with utility
2. Edit it to contain standardized evaluations
3. Copy it over to another, similar case
*** Additional sub-module =PyFoam.IPython=
The purpose of this submodule is to support
=pyFoamIPythonNotebook.py=. It has the classes
- Notebook :: read a file and interpret it as an
IPython-notebook. Do manipulations on this notebook
- PermanentStorage :: Implements permanent storage in an
IPython-notebook. Only works inside a notebook and allows
only one instance at once. Passing the data from the notebook
(through JavaScript) to Python currently is a performance
bottleneck
- Case :: Convenience object that exposes the functionality of
some of the PyFoam-utilities through a simple interface
*** Additional sub-module =PyFoam.Wrappers=
Wraps popular Python-libraries to add functions that make it
easier to work with OpenFOAM-data.
Currently only one Wrapper is implemented:
**** =Pandas=-wrappers
This provides =PyFoamDataFrame= as a wrapper for =DataFrame=. The
functionality added is
- addData :: Conveniently add new data from different =Series=,
=DataFrames=. It is assumed that the index is the
same property (time or for samples the distance) but
with a different resolution. The indexes are joined
and missing data is interpolated
- integrate, validLength, weightedAverage :: uses the index as
the $x$-axis and calculates these properties for a
=Series=. =validLength= is the extent on which data is
defined (=weightedAverage= is basically =integrate= divided
by =validLength=). For =integrate= the trapezoid-rule is used
- describe :: adds the three above quantities to the regular
=describe=-command
** Enhancements to Utilities
*** =pyFoamSampleplot= has option to use index instead of time in filenames
The option =-index-instead-of-filename= switches this on. This
makes it easier to generate movies from the files
*** =pyFoamListCases.py= Allows addition of custom data
The option =--custom-data= now allows the specification of custom
data items. These are read from the =pickledData=-files and
displayed in the table like regular data items
*** Switch compiler versions
Now all utilities allow switching the compiler version (for
instance from =Gcc47= to =Gcc48=). The relevant options are
=--force-system-compiler=, =--force-openfoam-compiler= and
=--force-compiler=
*** =pyFoamVersion.py= reports the installed versions better
Now the location of the installations is reported as well
*** Offscreen rendering can be switched off in =pyFoamPVSnapshot.py=
This is a workaround where the writer produces a segmentation
fault
*** Write 3D-data in =pyFoamPVSnapshot.py=
In addition to writing out bitmaps allows writing out 3D-data (for
importing into other applications). Sources can be selected by name
*** Added capabilities to =pyFoamSTLUtility=
The utility can now also:
- erase selected patches
- merge selected patches into one
*** =pyFoamDecomposer.py= switches off function objects
This now automatically happens for OF-versions that support
it (2.0 and greater). They can be switched on again
*** =pyFoamCloneCase.py= clones more stuff
Files that are assumed to be used by =pyFoamPrepareCase.py= are
automatically added to the clone. This includes all files (and
directories) with the extensions =.sh=, =.template= and
=.org=. Also IPython notebooks (extension =.ipynb= are added)
** Enhancements to the Library
*** =BasicRunner= now can print the command line that is actually used
This should help with diagnosing problems with MPI etc.
Can be switched on in some utilities with =--echo-command-prefix=
*** =ClusterJob= now can live without a machinefile
Using the machine-file now can be switched off for job-schedulers
with a tight integration
*** Enhanced treatment of symlinks during cloning
If a item in the case itself is a symlink then it used to be a
copy of the file the symlink is pointing to. Now it is created as
a symlink to the target the original symlink. If the
=--follow-symlink=-option is used the old behaviour is used
(copying). In this case the option =noForceSymlink= in the
=Cloning=-section of the configuration can be used to change this
behaviour for selected files
*** =AnalyzedCommon= clears the =analyzed=-directory
The directory is cleared if it exits from a previous run.
*** =TimelineDirectory= is more tolerant
Used to fail if incompatible data types were used. Now ignores
them
*** Possibility of a subcommand-interface for utilities
The subclass =SubclassFoamOptionParser= now allows the parsing of
subclasses. The base class for utilities =PyFoamApplication= now
supports this as an option. As an example this is implemented in
=pyFoamSTLUtilities.py=
*** =STLUtility= accepts file-handles
The class checks whether arguments are filehandles and in this
case doesn't try to open a file for reading or writing but uses
the handle
*** =addClone= in =SolutionDirectory= accepts glob patterns
If no file matching the name is found it is assumed that this is a
glob-pattern and all matching files are added. This affects all
utilities that use that method (especially =pyFoamCloneCase.py=)
*** =execute= in =Utilities= allows specification of working directory and echoing of output
This method now allows the specification of a working
directory. Before executing the command the method changes to the
working directory. Afterwards it changes back to the regular
working directory.
There is also an option =echo= that immediately prints the output
to the screen
*** =rmtree= and =copytree= more tolerant
=rmtree= now also works if the "tree" is a file.
=copytree= now has a parameter =force= that allows removing the
destination directory if it exists
*** Enhanced support for booleans in the parser
Strings that are usually interpreted as boolean in OF-dictionaries
(for instance =on=, =yes=, ...) are now stored as a special type
that allows treating them like 'real' booleans.
For instance an expression =test no;= in a dictionary now allows
things like =if d['test']:= in the script
*** Application classes now allow specifying options as keyword parameters
Until now the options to be used had to be specified as a list of
string modeled on the way the command line looked like. This is
still possible. In addition it is now possible to specify these
things as keyword parameters on the command line. Rudimentary type
checking is done. The names of the parameters are generated from
the command line options: the =-= are removed and the words are
converted to CamelCase. So for instance =--list-custom-Regexp =
becomes =listCustomRegexp=. Also for switches like these a boolean
value has to be passed. So the correct usage in a call would be
~listCustomRegexp=True~.
*** =SolutionDirector= now can classify directories in the =postProcessing=-directory
A number of properties has been added that list data generated by
function objects:
- timelines :: timeline data (like =propes=) that can be
visualized by =pyFoamTimelinePlot.py=
- samples :: data from =set= (assuming it is in =raw=-format) that
can be processed by =pyFoamSamplePlot.py=
- surfaces :: data from =surface= (assumes =VTK=-format) that can
be used by =pyFoamSurfacePlot.py=
- distributions :: special cases of =sample= with distribution
data
These properties only list the subdirectories of the case with
that data
Additional properties are
- pickledData :: a list of pickled data files that are found
- pickledPlots :: list of found pickled plots
These lists are sorted in descending temporal order (newest first)
*** =pyFoamSamplePlot.py= now more flexible for distributions
Tries to determine the names of the values from the first line in
the files
*** =DictProxy= now has a =dict=-like =update=-method
This also allows enforcing string values
*** =FoamFileGenerator= automatically quotes strings
If strings are unquoted but contain characters that make it
illegal as a word then the string is quoted before output
*** Children of =FileBasis= now can be used with the =with=-statement
This mainly concerns =ParsedParameterFile=
* Version 0.6.2 - 2013-11-03
** Major changes
*** Use of =pandas=-library
Starting with this version the =pandas=-library is used for
data-analysis. When possible and appropriate classes return
=pandas=-objects. Currently these are:
- =CSVCollection=. With a call-operator this class returns the
collected data as a =DataFrame= of the collected data
- =SpreadsheetData= now has methods to return =Series= and
=DataFrame= objects
It is not necessary to install =pandas= if these classes are not
used (and even then most of their functionality works)
** Incompatibilities
*** Different separator for databases in CSV-files
The class =RunDatabase= (and therefor also the utility
=pyFoamDumpRunDatabaseToCSV.py=) now write as a separator for data
from sub-tables a =//= instead of the space. This especially means
that scripts that rely on a data-item =foo= in =analyzed= might
break because this is now called =analyzed//foo= instead of
=analyzed foo=. On the other hand this makes the names more
consistent and easier to parse as =//= is the saperator for other
levels of dictionaries
*** Change of independent variable name in sample data
Instead of =col0= this is now =coord=. This could cause problems
with scripts that use that column name in the resulting
=SpreadsheetData=-object
** Bugfixes
*** =pyFoamPackCase.py= does not handle symbolic links correctly
Symbolic links were copied as is and did not work correctly
afterwards. This is fixed. If the symbolic link is an absolute
path or points outside the case directory it is replaced with the
file it points to. Otherwise it is preserved as a symbolic link
*** =pyFoamPotentialRunner.py= not working with OpenFOAM 2.0 or newer
These versions require an entry =potentialFlow= in the
=fvSolution=-file instead of the old =SIMPLE=
*** =pyFoamListCase.py= fails with =controlDict= that use preprocessing
Fixed by first trying to read that with preprocessing. Without if
that fails
*** Cloning fails in symlink-mode if files are specified twice
Now using a =set= instead of a =list= makes sure that no file is
cloned twice
** Utilities
*** =pyFoamPotentialRunner.py= now allows removing of =functions= and =libs=
The utility now allows removing these entries in case that they
don't work with =potentialFoam=
*** The Runner-utilities now have more options for clearing
Some of the options of =pyFoamClearCase.py= for clearing cases
(for instance specifying additional files) have been ported to the
=Runner=-utilities. Also is the =postProcessing=-directory
removed by default
** Library
*** =SolutionDirectory= and =TimeDirectory= are more tolerant
If there are field files and their zipped counterpart than
instead of an error a warning *can* be given
*** =ClusterJob= now handles template files
A new method =templateFile= gets the name of a file which is
constructed from a template of the same name plus the extension
=.template=
*** Additional parameters in =ClusterJob=
The method =additionalParameters= can return a dictionary with
additional parameters
*** Custom data in directory easier accessible
In the written data in the sub-dictionary =analyzed= there is now
a subdictionary =Custom= with the values of the custom expressions
with the prefix =CustomXX_= removed. This means that values that
were available as
: data['Custom02_velCheck']['min']
are now available as
: data['Custom']['velCheck']['min']
The advantage is that the number part which was dependent on the
order the expressions were specified is now no longer necessary
(this should make scripts more stable)
The old notation is still available but deprecated
*** =SolverJob= now allows compression of output
The parameter =solverLogCompress= compresses the log-file while
writing it to disc. *Attention:* This may lead to corrupted
log-files if the run crashes
*** =PyFoamApplication=-class now allows quick access to data
The dictionary returned by =getData()= now allows access to all
the elements as attributes.
** New features/Utilities
*** Post-run hook that sends mail at the end of run
The hook-module =MailToAddress= sends a mail at the end of a
run. Prerequisite is an SMTP-Server that doesn't need
authentication
*** New utility =pyFoamCompressCases.py=
This utility goes through cases and compresses single files. The
cases can be searched recursively to.
Purpose of this utility is to shrink cases where
=writeCompression= was not turned on during the run
*** Paraview-module to read additional data
A new module =PyFoam.Paraview.Data= reads additional data usually
written by OpenFOAM. These are converted to =vtkArray= using the
following functions and can be used in =Programmable filters=:
- setSampleData :: reads the data from sampled sets
- setTimelineData :: reads data from a timeline directory
- setPlotData :: reads pickled plot data using =RedoPlot=
** Enhancements
*** =pyFoamRedoPlot.py= can plot in XKCD-mode
When used with the option =--implementation=xkcd= and version of
=matplotlib= that supports it is installed then plots are done in
the style of the webcomics [[http://xkcd.com]]
*** =pyFoamListCases.py= now displays disk usage in human readable form
If the disk usage of the cases is calculated then it is displayed
in human readable form (as KB, MB, GB or TB) for sizes larger than
one Kilobyte
*** =pyFoamClearCase.py= more flexible in selection of data to be removed
Options to be more flexible in removing data are added:
- keep-interval :: keep timesteps at a specified interval. For
instance =--keep-interval=0.1= will keep times
like $1$, $1.1$ etc but remove $1.05$
- keep-parallel :: this will not remove any times in the
=processor=-directories. Also are things like
=keep-last= now honored for processor directories
- remove-analyzed :: Remove the directories with the analyzed data
too. Old behavior was to remove them. Now they are kept by default
*** =pyFoamFromTemplate.py= automatically chooses template and default values
If an output file =foo= is specified and no template then the
utility looks for a file =foo.template= as a template.
If a file =foo.defaults= is present then this file is read and
used as default parameter values. Other specifications override
these defaults
*** =pyFoamDumpRunDatabaseToCSV.py= can disable standard-fields
Additional option =--disable-run-data=
*** =pyFoamDumpRunDatabaseToCSV.py= prints =pandas=-object
With the =-pandas-print=-option a =DataFrame= is generated and
printed
*** Better debugging with =ipdb=
If the =ipdb=-package (basically =pdb= with =IPython=-additions)
is installed then it is used. This gives additions like
tab-completion
*** Interactive shell after execution for utilities
The option =--interactive-after-execution= drops the user to an
interactive shell where the namespace can be inspected. If present
=IPython= will be used, otherwise the regular shell is used
*** Utilities that read quantitative data convert to =pandas=-data and/or =numpy=
This is mainly to be used on the interactive shell to do further
analysis or write this data out. The utilities are:
- pyDumpRunDatabaseToCSV.py :: add an item =dump= with the whole
data as a =DataFrame=
- pyFoamTimelinePlot.py :: add element =series= with all the data
as =Series= and =dataFrame= with the same data as a =DataFrame=
- pyFoamSamplePlot.py :: Like =pyFoamTimelinePlot.py=
- pyFoamRedoPlot.py :: Now can get series and the whole plot data
as pandas-objects
*** Utilities that read quantitative data write Excel files
The utilities =pyDumpRunDatabaseToCSV.py=,
=pyFoamTimelinePlot.py=, =pyFoamSamplePlot.py= and
=pyFoamRedoPlot.py= now have options to write Excel-files
*** Specify additional settings for =GnuPlot= in =customRegexp=
If an item in =customRegexp= has an item =gnuplotCommands= then
it is assumed that this is a list of strings which are executed
before the first plotting. For instance
: gnuplotCommands (
: "set format y '%.2e'"
: );
changes the number format on the y-axis
*** More flexible data specification for =pyFoamSamplePlot.py=
Instead of determining the names of the fields and lines form the
filenames it is now also possible to specify them through options.
The option =--is-distribution= is a shorthand that sets these
options for distribution files
*** =pyFoamSamplePlot.py= now allows specification of x-range
The range of the x-axis of the plots can either be set by
automatically scaling to the domains of all the data sets with
=--scale-domain= or by specifying them with =--domain-minimum= or
=--domain-maximum=.
These domains are set for *all* plots
* Version 0.6.1 - 2013-05-24
** Major changes
** Bugfixes
*** Restoring of =controlDict= after =write=
When activating an on-demand write the =constrolDict= was not
restored because the output-line about the file being read was not
recognized (due to a change in the output in recent
OF-versions). Now a number of different formats is recognized
*** Custom-plot type =slave= not working if no =master= defined
That plot-type needs a =master=. Fixed to fail if none is defined
*** =-list-only= did not correctly parse lists with a numeric prefix
This did affect all utilities that use that option and also calls
with =listOnly= to the library class
** Utilities
*** =pyFoamBuildHelper.py= now allow more than one action
If multiple actions like =--update= and =--build= are specified
they are executed in a sensible order (update before build etc)
*** Utilities warn if OpenFOAM-version is unset
If the environment variable that determines the OpenFOAM-version
is unset a warning is issued by the utilities
*** =pyFoamUpgradeDictionariesTo20.py= allows single files
If single file is specified then the action to transform it has
can be specified
*** =pyFoamUpgradeDictionariesTo20.py= transforms reaction-schemes
Now knows how to transform "old" reaction files (where the
=reactions=-entry was a list) to the new format (where it is a
dictionary). Only a limited number of reaction types is supported.
*** =pyFoamUpgradeDictionariesTo20.py= transforms thermophysical data
Now the old form of thermophysical data (lists) is transformed
into the new dictionary-form
*** =pyFoamCloneCase= now allows creating directory that symlinks to the original
Now with the option =--symlink-mode= instead of copying the
directories from the original new directories art created and
populated with symlinks to the files in the original. The depth
until which no symlinks to directories are created can be
specified. This allows the clone to share the configuration files
with the original
*** =pyFoamClearCase.py= now removes =postProcessing= and allows removal of additional files
The directory =postProcessing= is now automatically removed (can be
switched off with =--keep-postprocessing=). Also with the
=--additional=-option patterns with additional files to remove
can be specified.
*** Improvements to =pyFoamVersion.py=
- Now reports the location of the =python=-executable
- Reports locations of used libraries
*** Additional files automatically cloned
The files =Allrun=, =Allclean= and =0.org= are automatically
added during cloning as these are often used by the standard-utilities
*** =pyFoamDisplayBlockMesh.py= uses the same options for template format as =pyFoamFromTemplate.py=
This makes sure that templates are handled consistently and also
allows different delimiters in the =blockMeshDict.template=
** Library
*** Improvements in syntax of =ParsedParameterFile=
- Now the new relative scoping that was introduced in OF 2.2 is
supported
*** =Utilities=-class now function to find files matching a pattern
Added a function =find= that approxiamtes the =find=-command
*** VCS ignores more files
Some more patterns have been added that will be ignored in a
VSC-controlled case. All of them concerning files that PyFoam
creates during operation
** New features/Utilities
*** New Utility =pyFoamSymlinkToFile.py=
This utility replaces a symlink with a copy of the
file/directories it points to. To be used after a
=pyFoamCloneCase.py= in =--symlink-mode=
* Version 0.6.0 - 2013-03-14
** Major changes
*** Adaption to work with Python3
Sources are adapted so that =PyFoam= works with Python3 too. This
breaks support for Python 2.4 and earlier (possibly also Python
2.5)
Some of the Libraries in =PyFoam.ThirdParty= had to be adapted to
work with Python3:
- Gnuplot.py :: The original version 1.8 is quite old. It was
adapted with the help of the =six=-library (see
below) to work with Python2 and Python3 (inspired
by
[https://github.com/oblalex/gnuplot.py-py3k/commits/master]
which is a pure port to Python3 without backwards
compatibility)
*** New ThirdParty-Libraries
- six :: Library that helps supporting Python 2 and Python 3 in
the same source code. Currently version 1.2 from
[https://bitbucket.org/gutworth/six] is used
- pyratemp :: Templating library to support the new templating
format. Version 0.2.0 from
[http://www.simple-is-better.org/template/pyratemp.html]
is used
*** Porting to =Windows=
Port to allow running PyFoam on Windows was done by Bruno Santos
of blueCAPE (bruno.santos@bluecape.com.pt)
Patch was originally posted at
http://sourceforge.net/apps/mantisbt/openfoam-extend/view.php?id=166
*Note*: many of PyFoam's features are not yet fully functional on
Windows.
*** Experimental port to =pypy=
Sources are executed in =pypy= but it seems there are problems
with =numpy= and also with code like =for l in open(f).readlines()=
** Third-Party
*** Upgraded =ply= to 3.4
This brings virtually no changes. =README= with copyright
information has been added
** Infrastructure
*** Parameters can't be modified in =CTestRun= after initialization
This should help to avoid side-effects
*** Treat timeouts in the =MetaServer= right
Due to a previous workaround timeouts when collecting information
about new machines was not treated correctly
*** Add =execute=-method to =ClusterJob=
This allows the execution of a shell-script in the directory of
the case
*** Add possibility to run specific modules before or after the solver
These modules are found in =PyFoam.Infrastructure.RunHooks=. Two
concrete implementations:
- =PrintMessageHook= :: to print a text to the terminal
- =SendToWebservice= :: encode an URL and send it to a webservice
(example for =pushover.net= added)
Hooks are automatically instantiated from the configuration data
(examples are hardcoded))
*** Parameters added to the info about the run
The Runner-classes now have a parameter =parameters=. This data
(usually it would be a dictionary) is added verbatim to the run
info.
Most runner applications now have the possibility to add this
info.
Purpose of this facility is to identify different runs in the
database better.
*** Parameter handling in =ClusterJob= extended
Parameter values are now handed to the actual job. Also a
dictionary with parameters can be handed to the constructor and
will be used in the relevant callbacks
*** Run data written alongside =PickledPlots=
During the run whenever the =PickledPlots=-file is written a file
=pickledUnfinishedData= gets written. This has the current solver
data and is similar to =pickledData=.
Also a file =pickledStartData= gets written that has the data that
is available at the start of the run.
*** =BasicRunner= collects error and warning texts
The runner collects
- at every warning the next 20 lines of the output until a total
of 500 lines is reached (this avoids filling disk and memory if
the solver produces too many warnings)
- All output from an error message until the end
And stores them in the application data
** Library
*** =TemplateFile= now uses =pyratemp=
The class =TempalteFile= now uses an enhanced templating
engine. The old implementation is in the class
=TemplateFileOldFormat=
*** Clearer error message in Application-classes
If used as classes (not as utilities) these classes print the
class name instead of the calling utilities name
*** Output is only colored if it goes to the terminal
Error and warning messages don't decorate the output if it goes to
files or other non-terminal streams
*** =error=-method of application classes now raises an exception
An exception is now raised by =self.error()=. This makes it easier
to handle such errors if the application class is used. The
exception is passed up until there is a "real" application
*** =ParsedParameterFile= now knows how to handle binary files
When the format of a file is =binary= lists with a length prefix
are being read as binary blobs.
For reading the blobs a simple heuristics is used: a multiple of
the length in bytes is read. If the next character is a =)= and
the characters after that are a certain combination of characters
(newlines and =;=) then it is assumed that the blob has
ended. This may fail on certain occasions:
- if the combination of characters appears at these places
- if the objects inside the binary data are of different sizes
It would be hard to work around these restrictions without
reprogramming the full functionality of OpenFOAM
*** =LabledReSTTable= for more flexible table generation
New class in the =RestructuredTextHelper= allows more flexible
generation of tables. Items are added with =column= and =row= and
if these don't exist in the first row/column the table is extended
appropriately
*** Plotting classes now allow setting of =xlabel=
This is implemented for =Gnuplot= and =Matplotlib=. Default for
the label on the x-Axis is now "Time [s]"
** Utilities
*** =pyFoamFromTemplate.py= with new templating engine
The utility can now use the pyratemp-templating engine which
allows templates with loops, conditions and other fancy stuff
*** =pyFoamSamplePlot.py= allows using the reference data as basis for comparison
Instead of using the x-values from the original data the y-values
of the reference data can be used for comparing (with the
=--use-reference=-option)
Same for =pyFoamTimelimePlot.py=
*** Scaling and offsets are now used in plots of =pyFoamSamplePlot.py=
If scales not equal to $1$ and offsets not equal to $0$ are
specified they are used in the =gnuplot=-output
*** =pyFoamPrintData2DStatistics.py= prints relative average error
With the =--relative-average-error=-option
*** Enhancements to =pyFoamVersion.py=
- More tolerant if no library was found
- Reports the location of the PyFoam-Library
- Checks whether utility version is consistent the library found
*** =pyFoamRunner.py= allows hooks
Hooks can be added at the start and the end of a run
*** =pyFoamRedoPlots.py= supports range for plots
Added =-end= and =-start=-option to select a range that should be
plotted.
Currently not working with the Matplotlib-implementation (only gnuplot)
*** =pyFoamDisplayBlockMesh.py= no supports templates
If a file with values is specified then the utility assumes you're
editing a template file and will evaluate it before displaying it
*** =pyFoamCaseReport.py= is tolerant towards binary files
New switch that makes the parser treat files that are declared
=binary= in the header as if they were =ascii=
*** =pyFoamSamplePlot.py= and =pyFoamTimelinePlot.py= raise error if no plots are generated
This makes it easier to catch faulty specifications (or empty
timeline-files)
*** =pyFoamSurfacePlot.py= can wait for a key
An option =--wait= has been added that makes the utility wait
before displaying the next picture
*** =pyFoamEchoDictionary.py= is more flexible with binary files
Switch allows forcing it to read a binary File as an ASCII
*** All utilities now have a switch that starts the debugger even with syntax-errors
Previously the option =--interactive-debug= only started the
debugger if the error was *no* syntax error. This is still the
default behavior, but can be overruled
*** Utilities now can be killed with =USR1= and will give a traceback
The option =--catch-USR1-signal= now installs a signal-handler
that prints a traceback and finishes the run. If the interactive
debugger is enabled then it goes to the debugger-shell.
Option =--keyboard-interrupt-trace= triggers the same behaviour
for keyboard interrupts with =<Ctrl>-C=
*** Switch to switch on *all* debug options
For the purpose of developing a switch =--i-am-a-developer= has
been added.
*** Plotting utilities now allow specification of x-Axis label
With the option =xlabel= in the =customRegexp=-file the label on
the x-axis of the plot can be changed. Setting =ylabel= and
=y2label= (for the secondary axis) was already possible
*** Metrics and compare for =pyFoamTimelinePlot.py= and =pyFoamSamplePlot.py= support time ranges
Now the options =--min-time= and =--max-time= are supported by
=--metrics= and =--compare=
*** =pyFoamDisplayBlockMesh.py= allows graphical selection of blocks and patches
New addition by Marc Immer allows the graphical selection of
blocks and patches and adds them to the =blockMeshDict=
*** =pyFoamCloneCase.py= and =pyFoamPackCase.py= accept additional parameters
The file =LocalConfigPyFoam= is read by these utilities and if
there is a parameter =addItem= in the section =Cloning= defined
then these files are cloned/packed automatically (no user
specification required)
*** =pyFoamListCases.py= now calculates estimated end-times
Additional option to print the estimated end times. These can be
wrong if the case did not start from the =startTime= in the
=controlDict=.
Also now allows printing the end and the start-time according to
the =controlDict=
** New features
*** Different "phases" for multi-region solvers
Plots of type =phase= in =customRegexp= don't actually plot
anything. The set a phase-name that is used for subsequent values
(for instance to distinguish the different residuals)
*** =pyFoamChangeBoundaryType.py= allows selection of region and time
Options =--region= and =--time-directory= added that allow
selecting different =boundary=-files
*** New class for storing case data in a sqlite-database and associated utilities
The class =RunDatabase= stores the data from runs. Utility
=pyFoamAddCaseDataToDatabase.py= is one way to populate the
database. =pyFoamDumpRunDatabaseToCSV.py= allows dumping that
data to a file for further processing (in a spreadsheet for
instance)
Database can also be populated using a special post-run hook
** Bugfixes
*** Only binary packages of 1.x were found
Pattern had to start with 1 (now every digit is possible))
*** Option group /Regular expressions/ was listed twice
No harm done. But fixed
*** =--clear=-option for =pyFoamDecompose.py= not working
Reason was that =rmtree= does not allow wildcards. Fixed
*** =pyFoamDisplayBlockmesh.py= not working with variable substitution
The =DictRedirect= would not convert to float. Fixed. Although it
might happen again for other data types
*** Option =--function-object-data= of =pyFoamClearCase.py= not working with directories
The option was only implemented for the list-form of the
=functions= entry in =controlDict=
Now fixed to also work with the dictionary-form
*** =nonuniform= of length 0 not correctly printed
Seems like the length was interpreted as the name of the
list. Fixed
*** Building of pseudocases with =pyFoamRunner.py= broken
Only worked if no region was specified (= not at all). Fixed