"How to create TOS components" Tutorial : Part 1

It must be a rainy day today there.
No? You mean you really want to learn how to create Components for Talend? Ehm.. ok then, I must admint I did want too some time ago, but rest assured : it was definitely raining like hell outside, my girlfriend was about 1000 Km far from me and my cats were sleeping all the time!

Ok, ok, did not want to scare you, sorry, actually building components for TOS can be fun after you nailed a few basic concepts.
The issue I found was that those basic concepts were pretty well hidden in the already scarce documentation available, or at least this is the way I felt when I started.
Even when I went through the official training to pass the certification test I felt that a lot of "grey areas" were left in the whole matter, but eventually I got around it.
How did I make it? Mostly by reverse engineering the existing components, yup, that's the beauty of open source, all the knowledge lays in front of you, if you can read easily uncommented java - jet  source code that uses objects you never heard before.
The purpose of this tutorial is to spare you some of that painful activity, or at least it would have spared it to me.

Let's get to the "meat" and skip all these personal considerations, shall we?

What knowledge should you have before starting? Some java knowledge I think is needed, you don't need to be a java guru, but you need to be able to read java code and to grasp the basic OOP concepts.

The component

You probably already realised what a component is :
Graphically it is an icon you drag from the palette to the project canvas, that's the easy part.
Technically what's behind that is a file folder which has the exact same name of the component, like "tWhateverStuffIamDoing" (I guess the starting "t" stands for Talend or  Transformation or Trouble... not sure, but as a standard, components have names that start with "t", I think I can live with that and with my ignorance about it) .

Where is this folder located?
Promise you will not laugh, it is in : TOS-xxxxxx (your insall dir) / plugins/org.talend.designer.components.localprovider_5.0.0.M1_r64233 (or whatever version you have), I guess a simpler "/components" was a bit too much to ask for.
That's the directory where you have the "standard" components.
There is also a directory named /org.talend.designer.components.ecosystem  and there you can find the additional components you eventually downloaded from the studio.
Enter in the localprovider one, there you will se a lot of folders starting with a "t"... behold, the components!

Is not...

At this point, at the beginning of my "Talend studies" I figured each component was finally compiled into a java class and placed in some kind of package called org.talend.designer... whatever.
That would be normal in java, no? Nobody said that the Talend guys went for the "normal" way, so I was terribly wrong, this is a very important basic concept :

a component is not a class.

You cannot even imagine how important this concept is, it will make many things more understandable later, things that normally would sound weird, as an example for the variable scope of the variables you define within your components.
Oh yeah, if you are not carefull, the lovely variable "i" that you use in most of your  loops  will be exactly the same in different components  in the job.
This would indeed not happen if each component was a class, but as we said, it's not, so some specific features have been implemented in TOS to overcome this issue.
Later, when you will use those features, and you will remmeber that a component is not  a class, you will understand why.
Gosh, I wish someone told me this at the beginning!

It is...

So,  now that we know what a component is not, can we define what it is?
Essentially it is (or it transforms into, later we will see how) a snippet of java code that is integrated within the TOS job.
Finally your job is a java class, whose java source code is generated taking into consideration  all the added components and  how they are linked, so you can picture each one of those components contributing with some  lines of java code.
When you run the job, this code is compiled and executed.
Now the funny thing is that this java code normally changes each time you alter a parameter in a component.
How's that possible?

The templates

We just said that a component finally generates some java code, that's normally referred to as "java output code" of the component.
Output of what? It is actually the output of a java template which you defined creating the component.
Quite confusing eh?
Yes, that's a bit strange I have to admit, but technically when you write a component  you are writing  java code (the JET Template) that is used to generate other java code, which is finally placed inside a job (a java class) and compiled.
It is more than a simple template, it is a real "software" that will be used to write the java output code, you will have loops, conditions etc.
All the parameters that a component has are interpreted by the template.

Parameters are not available to the java output code

that was something else a bit surprising for me at the beginning.
So, say you have a component which is used to connect to a Oracle database, this component will indeed need a number of parameters like hostname, username, password etc.
Surprisingly those parameters are NOT available to your java output code, they are just visible to the template, so the template will transform them into string constants and place them into the java output code when and where needed.
This will be something similar to :



Ok, the syntax is not exactly that one, we will get there, but at least you should have understood the concept.
Now you understand why the job java code changes each time you alter the value of a parameter in a  component.
That's  because those parameters affect the templates which will generate a different java code.
In the previous example we just "dumped" those parameters, but the template is a java program, so it can use those parameters in conditions, loops etc generating java output code completely different depending on the parameter values.
I tried to summarize the process in the picture below (you can click it to get a bigger version).

 

In the next part (unless the sun is really shining or my cat stops snoring on the chair close to me... or other things) I will explain the anatomy of a component, so we will start with some simple exercises as well.
Hope you enjoyed this one, if you have questions, ideas, complaints etc, be sure to let me know, you can use the feedback module on this site.

-> To Part 2