Not really. The research seems to be relatively novel, not a watered down version of a different study being repeated by children as your comment would suggest. My point was that the research just wasn't overly prestigious.
IPUMS, Univserity of Minnesota | Minneapolis, MN | Backend Web Developer | Full-time | Onsite | https://ipums.org/
IPUMS (https://www.ipums.org/) is a leader in the field of quantitative social science research and the largest disseminator of census and demographic data to the world’s academic research community. Or, to put it another way - we’re on a mission to gather, process, link and publish billions of records spanning hundreds of years and more than 100 countries so that demographers, historians, economists, environmental scientists, journalists, policymakers, and others around the globe can use the data to do amazing research and make the world a better place.
The position functions as a member of the data dissemination team and will have responsibility for design, implementation and support of software systems, including enhancing and adding new features to the IPUMS web applications and APIs as well as building standalone software systems. This position will report to the lead developer of the team. This position involves daily interaction with the rest of ISRDI IT, the ISRDI research staff, and other user communities.
Required Qualifications
* Bachelor’s degree. Two years of work experience in application/web/systems development with a related bachelor’s degree or four years of work experience in application/web/systems development with a non-related bachelor’s degree.
* API design and implementation
* Databases (relational or NoSQL)
Preferred Qualifications
* Containerization technologies such as Docker
* Service (or microservice) based architectures
* Developing applications in a Linux environment
* Working with and transforming large amounts of data
* The Ruby on Rails Framework
* Application server technologies such as NGINX and Passenger
* Message queues such as RabbitMQ and Kafka
* Elasticsearch
* Data Analytics and B.I. experience
IPUMS, Univserity of Minnesota | Minneapolis, MN | Backend Web Developer | Full-time | Onsite | https://ipums.org/
IPUMS (https://www.ipums.org/) is a leader in the field of quantitative social science research and the largest disseminator of census and demographic data to the world’s academic research community. Or, to put it another way - we’re on a mission to gather, process, link and publish billions of records spanning hundreds of years and more than 100 countries so that demographers, historians, economists, environmental scientists, journalists, policymakers, and others around the globe can use the data to do amazing research and make the world a better place.
The position functions as a member of the data dissemination team and will have responsibility for design, implementation and support of software systems, including enhancing and adding new features to the IPUMS web applications and APIs as well as building standalone software systems. This position will report to the lead developer of the team. This position involves daily interaction with the rest of ISRDI IT, the ISRDI research staff, and other user communities.
Required Qualifications
* Bachelor’s degree. Two years of work experience in application/web/systems development with a related bachelor’s degree or four years of work experience in application/web/systems development with a non-related bachelor’s degree.
* API design and implementation
* Databases (relational or NoSQL)
Preferred Qualifications
* Containerization technologies such as Docker
* Service (or microservice) based architectures
* Developing applications in a Linux environment
* Working with and transforming large amounts of data
* The Ruby on Rails Framework
* Application server technologies such as NGINX and Passenger
* Message queues such as RabbitMQ and Kafka
* Elasticsearch
* Data Analytics and B.I. experience
The decisions you are trying to make seem to be based more off of anxiety and/or fear rather than a passion for what you want to be doing with your life. Try to find out what you would enjoy doing with your current skillset and spend some time figuring out a way to make that happen. A MSc in CS opens doors to many industries as well academia, so you have tons of options to choose from.
You probably won't regret spending the next few years of your life doing something that you think is interesting. There is, however, a chance that you will regret spending the next few years worrying about if your future self will regret those years.
I’m not buying this. I ride the light rail in Minneapolis a fair amount and others on it don’t look or act rich at all. In fact the express bus I frequently take seems to be full of more affluent people than the light rail.
Whimsical inventiveness is the one thing that keeps me enjoying the art of programming.
There's a tension in programming between being verbosity and ingenuity. More flexible languages and programmers general favor ingenuity, while rigid ones favor verbosity. When people complain about the "magic" in Rails, it's usually because it's a pattern that is new to them, and they would have preferred an older pattern even if it was a more verbose one. Which is fine, but it is a preference amidst a large spectrum of tradeoffs. Unfortunately, many people tend to not see most tradeoffs in software engineering and try to come up with as many reasons as possible for why their way is right and the other way is trivial, stupid, cute, or useless.
> "Whimsical inventiveness is the one thing that keeps me enjoying the art of programming."
A personality difference then. Reading clever code does nothing for me. I'm much happier digging around hulking layered codebases that may look ugly but do something valuable for the user.
I agree that the cuteness of a programming language is just a matter of taste and habit. I'm not going to pretend my criticism was anything deeper than an expression of why I personally dislike having to work in Ruby.
I think there's more to it. Ruby, to me at least, seems objectively more difficult to understand intuitively when scanning the code. Maybe it's just a superficial lack of knowledge on my part, but my brain makes base assumptions on how code should work, and whenever something magical happens, it's cool, but slows down the cognitive understanding enough that it seems to me to be a net negative. There's a crossover point where verbosness becomes too much to grok as well. Maybe spending more time in the language raises the bar enough to then become massively more productive. I'm assuming this is the case with Ruby, but I've not given it enough time to make that determination. Python, on the other hand, is immediately understandable to me, and I can jump in and be productive with no previous experience in the language and just an open webpage to the documentation. To each their own, but this is a continuing question for me. The perfect example I think is Perl 6. To me, when I look at everything they're doing in Perl 6, it really does look like the next generation in languages. It's got so many good ideas and insanely helpful constructs. However I'm not so sure if the ability to do things cleverly is a benefit. I look back at Python and see how simple it is and how productive I can be in it immediately and ask myself.. what's the catch. I'd like to use these complex languages for fun's sake, but the alternative is very, very, intuitive.
> Ruby, to me at least, seems objectively more difficult to understand intuitively when scanning the code.
Reading Ruby code always reminds me a bit of trying to understand how xmonad configs work.
E.g. the Sequel example:
require 'sequel'
Nitpick: IMHO import-operators should operate on names not strings, because they essentially are equivalent to "foo := $magic".
DB = Sequel.sqlite # memory database, requires sqlite3
Why isn't this a function call? How would I pass options, e.g. the file name here? Why is it Sequel. here, but "sequel" above? Does "require" dump a bunch of unspecified names in my namespace?
DB.create_table :items do
primary_key :id
String :name
Float :price
end
What kind of construct is this? Where do all these names ("primary_key", "String", "Float") come from? They were never imported. Is this like ":name => foo" but because that doesn't nest we now have ":name do ... end" for nesting dictionaries?
I think this is just the usual lack of familiarity that you'll encounter with any language that has any non-C-like syntax.
The do...end bits are "blocks" and are the most powerful feature in Ruby, not just because of what they do but also because how they do it. Similarly for other comments.
When I first looked at Rust code it looked weird to me too but once you are familiar with the language a little bit it makes sense. The only thing special about Ruby here is that it allows you to be very terse.
Reading clever code does nothing for me. I'm much happier digging around hulking layered codebases that may look ugly but do something valuable for the user.
I don't think these need to be at odds with each other. Just because code is useful doesn't mean it needs to be hard to read.
What made me switch to Ruby was after I as an experiment rewrote a queuing middleware server that was about 5k lines in C, into about 500 lines of Ruby that had more functionality. It was slower, sure, but it drove CPU load from 1% to 10% of a single core before we hit disk IO limits, so it just didn't matter. What did matter was the improved maintainability of simplifying the codebase that much..
The point being that often the "hulking layered codebase" doesn't need to be.
Blatant plug: for large scale analysis on census and survey data I'd recommend checking out nhgis.org. It lets you download the large pieces of demographic data that you may be interested in. Data harmonized across time is also available to investigate trends.
Note: We don't have the 5-year 2017 ACS data yet (it takes about 2 weeks for us to ingest the data once it comes out).
I've used ACS and IPUMS data previously for research while at university. What are your thoughts on the proposed "differential privacy" changes [0] coming this way? Is it a "sky is falling" kind of situation?
It's not really my area of expertise, but from what I understand it will severely limit what policy and demographic research can be done with the data. There's a working paper on the page you linked to which has more information: https://assets.ipums.org/_files/mpc/MPC-Working-Paper-2018-6....
IPUMS is a leader in the field of quantitative social science research and the largest disseminator of census and demographic data to the world’s academic research community. Or, to put it another way - we’re on a mission to gather, process, link and publish billions of records spanning hundreds of years and more than 100 countries to demographers, historians, economists, environmental scientists, journalists, policymakers, and others around the globe, who then use the data to do amazing research and make the world a better place.
We are currently seeking a software developer to join our data linking team that is working on large scale data manipulation and creating highly performant distributed software systems. In this role, you will be developing software that links records representing the same individual and their family relationships across every US Census from 1850 - 1940. There are no consistent identifiers across historical censuses, so sophisticated matching algorithms will be required. You will be building on research and IT effort from the past decade to create a dataset that, once released, will enable critical new avenues of research for demographers and historians. This work will also lead to additional data linking opportunities for datasets from around the world and throughout history. You will be working in close collaboration with expert historians, demographers, and data scientists. We use Apache Spark (PySpark and SparkSQL) for a majority of our data processing, with C++ in critical areas. This is an excellent opportunity to grow your career with a cutting -edge software shop at the University of Minnesota and contribute to work that has impact around the globe.
Required Skills: BA/BS degree required. Two years of work experience in the areas of application/web/systems development with a related BA/BS degree, or four years of work experience in the areas of application/web/systems development with a non-related BA/BS degree.
This role requires technical experience and proficiency with:
- Experience developing software for high performance and/or distributed computing
- Experience with databases (relational or NoSQL)
- Large dataset manipulation
- Linux/UNIX operating systems, including command-line
I really like Sandi Metz as a speaker, and am pretty sure she has thought about these problems much deeper than I have. However, I often find myself wondering if most of the problems she is trying to solve would go away if programmers spent more time thinking about their code.
The case filled code that she is describing seems to be a result of the programmers not fully grasping the purpose of the code, and being unable to tell if the current abstraction is fitting. I understand that deadlines and the sunk cost fallacy play a factor, but, at least for me, finding the right abstractions / architecture is most satisfying part of coding! Shouldn’t that be what these programmers are focusing on in the first place?
