Want to understand how JPEG works a bit more?

[peterb]

And of course Fourier analysis tells us that we can never have a perfect square wave because you would need infinite bandwidth.

(I'm just wondering whether this thread would be better placed in GD...?)

[TheAnimus]

WRT the PIC assembly, you must have a lot more staying power than I. I tend to look at my code in LLVM assembly, see it's inefficient, say to myself: "I could really improve these things..." then realise it feels like too much effort. (I love autovectorisers, the fewer SIMD intrinsics I have to use the better). Having never had to program in assembly the amount of work involved seems like a pain, especially handling something like a DFT matrix...

So this is a trip down memory lane. First off, I must admit that I'm now old (I only just turned thirty, you've no idea how much booze I've drunk to make myself OK with the fact I'm old, I hope you are happy making me feel older!) because LLVM didn't exist for me. That's right, I'm too old. For me it was just MASM.

However this isn't really a problem, because I actually really like the PIC16 microchip instruction set. I mean it just felt clean. Before I got very drunk with a certain mod who changed my subtitle, I used to have the no-op code in binary as it was kind of beautiful. He thought that my current subtitle better suited me, he might be right!

You know I've not written any for years. It's been 6 years. A friend of mine has all my 'kit' for this, my ICD and all my chips. I'm shooting him an email now damnit. Hmmm, but then I like the effortlessness of my .Net Fez Spider toy. Oh well back to work tomorrow writing line of business software in high level languages I guess!

(I'm just wondering whether this thread would be better placed in GD...?)

Hmm you might be right, I posted it here because I thought such an excellent video, which could help anyone learn from scratch, would be of benefit to people who take photographs understanding why the lossfull compression works the way it does. If you think it's worth moving over to a larger audience, then I'm sure the video's author would be stoked about more audience, and please feel free to edit something into my first post about why it's worth the time!

[Goobley]

This is fairly OT, but I've written it now:
I've worked with arduino's and PIC's etc but in most applications I'm happy enough to write it in C and just cross-compile. For most projects computer power is a lot cheaper than my time these days.

LLVM certainly isn't the oldest backend I've worked with, it was just the first time I started looking into assembly. I started programming with my Dad in the Classic Mac OS days. First Pascal on System 7 and then moving to CodeWarrior C++ after about System 8.1/5 (although a bit of Pascal was always useful for some of the OS features). My Dad was never a trained programmer, he taught himself and was a firm believer that the days of assembly languages were behind us. I took a fairly long hiatus from programming before coming back to High-Performance stuff before going to uni.

Being that I'm interested in HPC for simulations and game development etc, asm seems like something I should deal with from time to time. Then again the state of our modern physics students in programming is shocking. I took a 2nd year C on Linux course aimed at those continuing in theoretical physics (but useful for physicists in general) and it had one of the highest drop out rates I've seen in a single course...

It might be good to spend some time with some of the nicer asm languages, possibly work through Knuth's art of computer programming... should also go back and spend more time on Haskell... maybe learn more about how this reliable "business" software is written... but they're all projects that take a lot of time, which is especially hard to find as I'm 40k LoC deep into my own game after having a disagreement with Unity.

When it comes to it, I wouldn't say that you're old, just have the benefit of a few more years experience. At my age (22 this year) everyone is looking for this experience to actually get into employment at some point (leaving aside the fact that physicists are completely unemployable ). Whatever your job, experimenting with fun projects like that could make a real difference later, in the same way that if someone asks about other things I've worked on I'm quite happy to tell them that I can lay a concrete screed and roughly plaster a wall. For a good while yet you have many benefits over us younger ones!

[peterb]

I've moved this from the photography forum as the topic may be of more interest to a wider audience. The principles of compression have wider application than just JPEG compression. Don't be daunted by the maths, the narratives by the post contributors explain it well!

[DanceswithUnix]

That's right, I'm too old. For me it was just MASM.

MASM? pah youngsters

For most projects computer power is a lot cheaper than my time these days.

I think there is a better argument: You can often get more out of high level optimisations than low level. Converting to assembler might get you up to double the performance if you are lucky (like encryption kernels) but more typically IME only about 20% faster. A high level optimisation might get you a magnitude faster by sorting things, moving stuff to the right place, adding hash bins, whatever it takes. Once you convert code into assembler, it becomes harder to read and harder to modify simply because it is about three times more lines of code than the equivalent 'C', so spotting and adding those high level algorithmic tweaks becomes harder.

Edit to add: I like that how jpegs work is best described with the help of a GIF in post #9 Yeah I know the mpg video compression builds on jpg principles, but that still made me smile.

[santa claus]

You can lead a horse to water, but you can't make them think. If you don't want to bother understanding something, you can never be in a position to say if it was worth learning or not. There are many things I've researched which have no practical application, no 'benefit' but the journey was still a worthwhile endeavour. You can't know how something may or may not benefit you, until you know that something.

Even if after learning it, it's not applicable to you doesn't mean it's a good idea to go around spouting a falsehood whilst claiming that the education for correcting it is boring. I think it's wrong to judge people for being ignorant. But wilful ignorance, well that is damned stupid.

Ouch, little bit harsh mate . From only very infrequent visits to the photography sub-forum it's pretty clear that Bobster's skill is in producing some amazing images and I think he rightly feels his time is best devoted to the things he is brilliantly creative at.

There is so much we all don't know; and there are many things I still don't understand (black holes) even when I read about them! I enjoyed the vid btw.

[TheAnimus]

Ouch, little bit harsh mate .

My second comment? Or my first one?

I don't mind ignorance at all, we are all ignorant, what I dislike is wilful ignorance, the desire to not learn. This can then be combined into one of my major pet hates, which is when someone openly spouts something incorrect, whilst simultaneously being disparaging towards improving understanding of that subject matter.

If you remember in black hole thread, I think a lot of this is an attitude due to the 'damage' done to peoples behaviour towards learning in school. I am honestly deeply concerned about this as to how it will affect our society going forward. A lot of our current living standards are because people can earn say £6 per hour here, consuming goods made by people earning pence. But things are changing, having spent time with school children in developing countries such as Thailand, Vietnam or China the attitude towards learning is very different, almost dangerously so when you get to South Korea. Children have to study have to learn, have to achieve academic prowess. Our lifestyles for the last few decades have been boosted thanks to this 'geo arbitrage'. Sure Keynes identified the gains of efficiency, but we've certainly moved towards a split society, those whose actions carry a multiplier effect, and those who don't, we will have an inequality gap, the way to address that is to prioritise learning more. The good news is we've never, ever in the whole of human history had such easy access to knowledge and learning, knowledge of whatever kind that interests you best. But if we don't get our act together as a nation, put much more of an emphasis on not just training, but inspiring people to educate themselves our society will suffer greatly.

[Ulti]

That was an interesting video but what really amazed me was how smoothly and clearly the guy explained everything.

It's always interesting to see how maths is pretty much in everything - the people who come up with these applications of maths are very smart.

[santa claus]

My second comment? Or my first one?

I don't mind ignorance at all, we are all ignorant, what I dislike is wilful ignorance, the desire to not learn. This can then be combined into one of my major pet hates, which is when someone openly spouts something incorrect, whilst simultaneously being disparaging towards improving understanding of that subject matter.

If you remember in black hole thread, I think a lot of this is an attitude due to the 'damage' done to peoples behaviour towards learning in school. I am honestly deeply concerned about this as to how it will affect our society going forward. A lot of our current living standards are because people can earn say £6 per hour here, consuming goods made by people earning pence. But things are changing, having spent time with school children in developing countries such as Thailand, Vietnam or China the attitude towards learning is very different, almost dangerously so when you get to South Korea. Children have to study have to learn, have to achieve academic prowess. Our lifestyles for the last few decades have been boosted thanks to this 'geo arbitrage'. Sure Keynes identified the gains of efficiency, but we've certainly moved towards a split society, those whose actions carry a multiplier effect, and those who don't, we will have an inequality gap, the way to address that is to prioritise learning more. The good news is we've never, ever in the whole of human history had such easy access to knowledge and learning, knowledge of whatever kind that interests you best. But if we don't get our act together as a nation, put much more of an emphasis on not just training, but inspiring people to educate themselves our society will suffer greatly.

An evidently passionately held view and one with which I don't disagree. But it does take the additional information to better understand where you are coming from.

[yamangman]

I'm sure Bob doesn't need a hand here but I shall plunge in...

Let's assume all that spiel is true. What's the assertion, Bob doesn't have this multiplier effect because he got bored reading about waves? No. It just leaves you being incredibly rude to him more than once.

(Cue paragraphs about not understanding people who are wilfully rude and prophesy the downfall of society because people are).

[Zak33]

Enough already.

Bob's input was harsh and Animus retort harsher.

In neither case does Jpeg explanation improve.

Move on.....nothing uncompressed to see here......

[TheAnimus]

It just leaves you being incredibly rude to him more than once.

Hands up anyone on here that I've not been incredibly rude too more than once? Only a really stupid fool will raise their hand....

But in all seriousness, I'll never be polite to someone who states something which is wrong because we all know:



That with a desire to shun learning, whilst being wrong, deserves no respect at all, if they push forward an attitude, they can get a response, but it might not be equal or opposing

I've said before many times that Bob is clearly a talented photographer, complemented many of his shots, would learning any of this make them look worse? Would it make them look better? Who knows, well those who understand these principles. It reminds me of font kerning, another great XKCD:

Many people will not notice it. But if you do notice it, it jumps out, screaming.

I currently am using a couple of very high resolution, good colour rendition U3415W panels (highly recommend), things such as compression become much more apparent at this DPI with this colour fidelity. I've found this quite an interesting thing, as lots of photos I'd taken years ago with my first DSLR simply don't cut the mustard any more. The high resolution is amazing at showing certain kinds of noise and compression behaviours. In fact I've only one image that looks remotely OK on it that was taken with my k20d which didn't have a great sensor, a lot of these issues can be fixed by reprocessing as they are linked to the decoding of the raw file.