Current Events > I don't think people are fundamentally prepared for how modern modern C++ is

(+) Yeah! (+)

Everybody knows the best language is espanol
---

scar the 1 posted...

Oh well there are most likely tons of calculations you want to program. And you'll probably need some program to shuffle data from one function to another, and to make plots, etc. It'll be really useful, especially once you figure out what you're actually going to be doing in your work.

Seems like C++ would be a silly programming language to use for data analysis, tbh.

Python, R, MATLAB, all seem more suitable.
---

scar the 1 posted...

How do you produce the data that you plot?

Experimentally

With the occasional transformation from the variable I measured into the variable I want to plot.
---

C++ is an amazing scientific language. Most scientists just don't care about programming enough to get good at it.
---

COVxy posted...

scar the 1 posted...

Oh well there are most likely tons of calculations you want to program. And you'll probably need some program to shuffle data from one function to another, and to make plots, etc. It'll be really useful, especially once you figure out what you're actually going to be doing in your work.

Seems like C++ would be a silly programming language to use for data analysis, tbh.

Python, R, MATLAB, all seem more suitable.

The lecturer for the course comes from the chair for fluid dynamics

They tend to use numerical data over experimental
---

COVxy posted...

scar the 1 posted...

Oh well there are most likely tons of calculations you want to program. And you'll probably need some program to shuffle data from one function to another, and to make plots, etc. It'll be really useful, especially once you figure out what you're actually going to be doing in your work.

Seems like C++ would be a silly programming language to use for data analysis, tbh.

Python, R, MATLAB, all seem more suitable.

I agree. However, what I've noticed at our university is that when students are only taught MATLAB, they often don't get very strong programming skills. So getting a solid foundation before you move on to the most practical language for your domain isn't necessarily a bad thing, and C/C++ is good for that.
---

teepan95 posted...

scar the 1 posted...

How do you produce the data that you plot?

Experimentally

With the occasional transformation from the variable I measured into the variable I want to plot.

Will you ever get to the point where the data is too much to collect by hand?
---

scar the 1 posted...

teepan95 posted...

scar the 1 posted...

How do you produce the data that you plot?

Experimentally

With the occasional transformation from the variable I measured into the variable I want to plot.

Will you ever get to the point where the data is too much to collect by hand?

I'd like to be prepared for the possibility.
---

teepan95 posted...

scar the 1 posted...

teepan95 posted...

scar the 1 posted...

How do you produce the data that you plot?

Experimentally

With the occasional transformation from the variable I measured into the variable I want to plot.

Will you ever get to the point where the data is too much to collect by hand?

I'd like to be prepared for the possibility.

you might want to use a database. There is almost zero chance that you would be able to get a custom application that will do what you want faster / better / with less bugs than a good RDBMS.
---

scar the 1 posted...

COVxy posted...

scar the 1 posted...

Oh well there are most likely tons of calculations you want to program. And you'll probably need some program to shuffle data from one function to another, and to make plots, etc. It'll be really useful, especially once you figure out what you're actually going to be doing in your work.

Seems like C++ would be a silly programming language to use for data analysis, tbh.

Python, R, MATLAB, all seem more suitable.

I agree. However, what I've noticed at our university is that when students are only taught MATLAB, they often don't get very strong programming skills. So getting a solid foundation before you move on to the most practical language for your domain isn't necessarily a bad thing, and C/C++ is good for that.

Do you really need a strong foundation in low level programming problems to be a good scientific programmer?

In my experience, not really. You need to understand algorithms at the data level. I'd much rather my program be .02% slower due to whatever automated memory management is going on under the hood, than to deal with memory management directly.

Even across big data, the inefficiencies of higher order languages usually don't create enough bother for one to want to understand pointers.
---

Damn, that's a sad statement.
---

ChromaticAngel posted...

you might want to use a database. There is almost zero chance that you would be able to get a custom application that will do what you want faster / better / with less bugs than a good RDBMS.

That really doesn't sound like something you'd want for data analysis, unless you're setting up a super serious processing pipeline.
---

ChromaticAngel posted...

teepan95 posted...

scar the 1 posted...

teepan95 posted...

scar the 1 posted...

How do you produce the data that you plot?

Experimentally

With the occasional transformation from the variable I measured into the variable I want to plot.

Will you ever get to the point where the data is too much to collect by hand?

I'd like to be prepared for the possibility.

you might want to use a database. There is almost zero chance that you would be able to get a custom application that will do what you want faster / better / with less bugs than a good RDBMS.

I'm guessing the DB stands for database?
---

My guess would be something like Relational Database Management Software/System.
---

scar the 1 posted...

ChromaticAngel posted...

you might want to use a database. There is almost zero chance that you would be able to get a custom application that will do what you want faster / better / with less bugs than a good RDBMS.

That really doesn't sound like something you'd want for data analysis, unless you're setting up a super serious processing pipeline.

MSSQL Server comes with SSRS which you can use for data analysis as well as custom ad-hoc data analysis that you can fine tune however you like by writing TSQL

Most people use them for high volumes of data but they're also fantastic for low volumes of data, and MSSQL Server Express is free.

teepan95 posted...

I'm guessing the DB stands for database?

Relational DataBase Management System
---

ChromaticAngel posted...

MSSQL Server comes with SSRS which you can use for data analysis as well as custom ad-hoc data analysis that you can fine tune however you like by writing TSQL

Most people use them for high volumes of data but they're also fantastic for low volumes of data, and MSSQL Server Express is free.

Still feels kinda weird to do scientific stuff in SQL. I would consider using it for storage, though. Getting to a point now where I suddenly handle kinda large volumes of data.
---

Btw, if you haven't already tried using it yet, give Boost.Spirit a try. It's an absolutely amazing library for building complex parsers and for generating strings as well.
---

MutantJohn posted...

Damn, that's a sad statement.

Not really. It's sort of like premature optimization, it would really be a waste of people's time for the most part.
---

as far as C++ has come, it's still held back by the lack of a proper module system. the preprocessor is downright prehistoric, and template definitions being limited to header files makes compilation time skyrocket

COVxy posted...

MutantJohn posted...

Damn, that's a sad statement.

Not really. It's sort of like premature optimization, it would really be a waste of people's time for the most part.

I was speaking to this bio grad student and they said their stuff took 2 weeks on the compute cluster the university had setup. This was most likely because it was all written in R. Which is absurd. CUDA is really popular in the bio world so I'm curious how parallelized her code could've been.


Keep in mind too, scientific computing is actually quite the specialty. C++ takes years to learn effectively and even then, new corner cases will greet you along the way.

treewojima posted...

as far as C++ has come, it's still held back by the lack of a proper module system. the preprocessor is downright prehistoric, and template definitions being limited to header files makes compilation time skyrocket

I agree. Tools like CMake help. MSVC supports modules now though (to some degree) so there's steps being taken. I also agree about the template definitions as well. It creates some weird design sometimes.
---

MutantJohn posted...

so I'm curious how parallelized her code could've been.

There's parallelization options in both Matlab and Python, Idk about R.

I've had things take weeks to process, but the data is incredibly large, and some of the analyses are complex.
---

there's also the rather inconvenient fact that there is no standardized C++ ABI. it just so happens that we all either use MSVC-style or GNU-style mangling. this makes interfacing other languages with C++ such a pain in the ass that the end result is always them saying "if you need to call out to an external library, just use C"

dang you guys know way more than me about this shit
---

treewojima posted...

there's also the rather inconvenient fact that there is no standardized C++ ABI. it just so happens that we all either use MSVC-style or GNU-style mangling. this makes interfacing other languages with C++ such a pain in the ass that the end result is always them saying "if you need to call out to an external library, just use C"

Yeah, I recently learned about this as well :P

Eh, it's okay. It's not like it's hard to create C APIs from C++ ones and vice versa. It sucks. It most definitely sucks. But so far it's not debilitating to integrating the language into other ones.

Though truth be told, I like the micro-service approach and separating things at the process-level.
---

treewojima posted...

as far as C++ has come, it's still held back by the lack of a proper module system. the preprocessor is downright prehistoric, and template definitions being limited to header files makes compilation time skyrocket

Is the preprocessor being prehistoric a bad thing? At least it HAS one. I don't recall the standard Java build tools having anything comparable, and interpreted languages generally read in definitions as-is.

The template bit is annoying but that's really because of the way C++ templates work. And you CAN work around it...it just requires explicit instantiation of every specialization you know is being used.
---

COVxy posted...

There's parallelization options in both Matlab and Python, Idk about R.

Although to use Python's parallelization options you need several instances of the interpreter running.

To the CUDA comment: GPGPU programming is indeed efficient, both MATLAB and Python have possibilities to dispatch stuff to GPU. However, things don't just magically speed up just because you run it with a GPU. A lot of problems are far too complex and would run slower, even if they can be parallelized easily.
---

Ha, except for me!
---