The true bottleneck of modern scientific computing in astronomy
The CPU power is growing exponentially in time following the Moore's law. Similar behaviour is shown by the amount data storage per price unit. However, in astronomy we do not see the exponentially growing avalanche of scientific results produced with this computational power. Where is the true bottleneck of the scientific computing? Astronomers as many other scientists, prefer to develop their computational codes and software systems (including DBMS solutions) themselves often having no coding skills, insufficient background in algorithms and computational science. As a result, (a) some well-known computational techniques become re-invented and/or inefficiently re-implemented; (b) obsolete software solutions and programming languages are used (e.g. Fortran-77); (c) the code cannot be understood by a third-party, i.e. it becomes unmaintainable in the absence of its author(s). In small projects, the same applies to data access solutions such as providing access to catalogues and databases. The scalability also suffers, because scientists can be incapable in implementing certain features (e.g. parallelisation) while a professional developer cannot approach the code. We will provide several examples of bad and good computing solutions in astronomy: (1) switching from a hand-made DB solution into PostgreSQL in the HyperLeda database; (2) optimisation of a stellar population synthesis algorithm for the GalMer project resulting in a performance gain of 10^5 compared to the original code; (3) bad access to relatively large catalogues stored as FITS binary tables from IDL; (4) good example of the Millennium simulation DBMS access; (5) scalable and non-scalable numerical simulations codes. The only solution we envisage is to improve the CS/algorithmic background of students in natural sciences as it is already starting to be done in some universities. The moment when the CPU power and/or data storage becomes the bottleneck of scientific computing will become the beginning of a new ``exponential'' scientific era.
Return to oral presentation list