Gunnar Beutner wrote:
as requested by Michael Friedrich I'm pushing the discussion that's so
far taken place on our internal team list to the icinga-devel list. For
anyone new to join this discussion here's a short introduction to what
this is all about:
i do think that such things are way off the hook to be kept
internal. those are major changes to be discussed, and not just
implemented and thrown at someone.
During the Icinga meet-up last weekend we've discussed some ideas for an
Icinga Core API and a way to deal with scalability issues in large-scale
i wasn't taking part during the meeting, so my opinion only
reflects what i was reading and reflecting in the past 3 days.
those things are 2 seperated portions to take care of.
1) a core api
* adding things like mklivestatus already proposed (GET)
* allowing the SET/POST bidirectional communication
* don't introduce external dependencies
* providing compatibility to existing addons
2) a distributed message queue system
* can be done either as (neb) module (mod_gearman!) or on the new
core api layer
* stays independent from the existing setup, can be provided "as
is" and loaded if demanded
* does the internal message queueing itsself and what might be on
the top layer
* doesn't bypass the core api
The basic idea revolves around using a message queue to decouple
components like service checks and API methods from the Icinga Core
while providing a unified interface to these components that can be used
either locally (i.e. in-process or via IPC) or remotely (via TCP). For
now we've been thinking about using ZeroMQ for this which is an
embeddable, (mostly) platform-agnostic message queue library written in C.
i'd love to hear opinions why exactly this should be zeromq and
not rabbitmq or other candicates like described here e.g.http://wiki.secondlife.com/wiki/Message_Queue_Evaluation_Notes
You can find an updated in-depth description of the proposed changes
involved in the MQ implementation at
https://wiki.icinga.org/display/Dev/Ideas+for+new+Core+API . I'd love to
hear your feedback about the design specifications which for now are
primarily being worked on by Ricardo and me.
summarizing the points i've already posted in recent internal
* the newly introduced core api should be used as poc for
classicui and integrated live search, providing simple GET hosts
in json and nothing more basically
* the libicinga is on top of the mq architecture, which then
remains rather independant of the underlaying core architecture.
so whatever you are wrapping around, you could still do it a layer
down below, if you know what you are doing .... "libicinga
is built on zeromq and people tend to use other technologies -
like gearman or livestatus wrappers for nagvis, nagiosbp and such.
the idea is *
to reinvent the wheel, but
stay compatible to existing solutions. addons devs won't jump onto
that just because it's cool and costs dev time."
* the proof of concept for distributed monitoring should
take place with the current neb architecture .... "proof of
concept as core module - take a look at mod_gearman, and replace
that parts with zeromq, realize the proof of concept as neb module
with NEB_CALLBACK_OVERRIDE or NEB_CALLBACK_CANCEL and threadinng
* the core api should be extendable. not only for icingamq but
also for ...
** idoutils replacement, with a cached worker
** add configs via api, store internally, provide a config editor
(or at least a community addon) like demanded over
1) design a core api capable of that (icingamq doesn't include that by
default as far as i have read)
2) allow simple configs to be added into the core, and still written
3) adopt a config data layout
4) build a custom config web onto that
5) integrate that web in classicui and icingaweb
* the core api as well as the icingamq design *must* entirely stay
out of the current 1.x tree and probably provided as package to be
tested, but not provided officially (and also not supported).
and tbh, this would be something for icinga 2.x where new features
can be introduced and some show stoppers can be deprecated whilst
introducing new things.
* the message format for icingamq must be discussed either on
icinga-devel or icinga-users mailinglists (or a public poll if
there was a decision), allowing the community to add their ideas
to a general RFC - which needs to be written in the first place.
* evaluate the core logic on the eventloop block for api fetches,
or threaded synchronisation. this includes feeding checkresults
* keep the code clean, follow the style guide and developer
guidelines. this includes doxygen and also further documentation /
DI (FH) Michael Friedrich
Vienna University Computer Center
Universitaetsstrasse 7 A-1010 Vienna, Austria
phone: +43 1 4277 14359
mobile: +43 664 60277 14359
fax: +43 1 4277 14338
Icinga Core & IDOUtils Developer