Christian Pühringer | 3 Oct 19:40 2011
Picon
Picon

Re: [Wikitech-l] Phonegap and Wikimedia mobile apps

Hi Arunesh,

Firstly, thank you again for doing the java port.

I've one question regarding the features of the port:
Is there or do you plan to add a feature for iterating over the index? (Similar to find/findByTitle in C++ zimlib)
This would be required to add some auto complete feature to the phonegap app.

Best regards,
Christian

Am 24.09.2011 14:42, schrieb Arunesh Mathur:
Yes, images are not compressed in ZIM files, as Emmanuel pointed out.

Also, I have tried decompressing a LZMA compressed file on an Android phone (HTC Wildfire to be precise), and the decompression speed is not a problem.

LZMA-Java is under frequent development by Lasse Collin, so we should make sure that the latest code is used.

On Sat, Sep 24, 2011 at 6:00 PM, Emmanuel Engelhart <emmanuel-dU3mOXAlhucBXFe83j6qeQ@public.gmane.org> wrote:
On 24/09/2011 14:24, Christian Pühringer wrote:
> The JAVA liblzma performance is pretty bad: To increase efficiency of
> compression in the zim-format articles (and also all
> other data like images) are stored in clusters. Cluster size is apparently about
> 1 MB. This implies that loading an article
> which is stored at the end of a cluster involves decompressing the complete
> cluster.

Images should not be compressed in ZIM files for the obvious reasons
they mainly are already compressed. This is the case for all ZIM files I
made. As far as I know this is also the case for Mediawiki:Collection
build ZIM files.

Emmanuel


_______________________________________________
dev-l mailing list
dev-l-rOD9jy6iqmlAfugRpC6u6w@public.gmane.org
https://intern.openzim.org/mailman/listinfo/dev-l




--
Best,

Arunesh Mathur
IV year, Undergraduate,
Department of Computer Science and Engineering,
National Institute of Technology Karnataka Surathkal,
India.



_______________________________________________ dev-l mailing list dev-l-rOD9jy6iqmlAfugRpC6u6w@public.gmane.org https://intern.openzim.org/mailman/listinfo/dev-l

<div>
    Hi Arunesh,<br><br>
    Firstly, thank you again for doing the java port.<br><br>
    I've one question regarding the features of the port: <br>
    Is there or do you plan to add a feature for iterating over the
    index? (Similar to find/findByTitle in C++ zimlib)<br>
    This would be required to add some auto complete feature to the
    phonegap app.<br><br>
    Best regards,<br>
    Christian<br><br>
    Am 24.09.2011 14:42, schrieb Arunesh Mathur:
    <blockquote cite="mid:CAF62jJDpVMgtYL3KwN6rNZSqBPhhgd7tDWBo0mdqHadtbFoKuQ@..." type="cite">Yes, images are not compressed in ZIM files, as
      Emmanuel pointed out.<br><br>
      Also, I have tried decompressing a LZMA compressed file on an
      Android phone (HTC Wildfire to be precise), and the decompression
      speed is not a problem. <br><br>
      LZMA-Java is under frequent development by Lasse Collin, so we
      should make sure that the latest code is used.<br><br><div class="gmail_quote">On Sat, Sep 24, 2011 at 6:00 PM, Emmanuel
        Engelhart <span dir="ltr">&lt;<a moz-do-not-send="true" href="mailto:emmanuel@...">emmanuel@...</a>&gt;</span>
        wrote:<br><blockquote class="gmail_quote">On
          24/09/2011 14:24, Christian P&uuml;hringer wrote:<br>
          &gt; The JAVA liblzma performance is pretty bad: To increase
          efficiency of<br>
          &gt; compression in the zim-format articles (and also all<br>
          &gt; other data like images) are stored in clusters. Cluster
          size is apparently about<br>
          &gt; 1 MB. This implies that loading an article<br>
          &gt; which is stored at the end of a cluster involves
          decompressing the complete<br>
          &gt; cluster.<br><br>
          Images should not be compressed in ZIM files for the obvious
          reasons<br>
          they mainly are already compressed. This is the case for all
          ZIM files I<br>
          made. As far as I know this is also the case for
          Mediawiki:Collection<br>
          build ZIM files.<br><br>
            Emmanuel<br><br><br>
          _______________________________________________<br>
          dev-l mailing list<br><a moz-do-not-send="true" href="mailto:dev-l@...">dev-l@...</a><br><a moz-do-not-send="true" href="https://intern.openzim.org/mailman/listinfo/dev-l" target="_blank">https://intern.openzim.org/mailman/listinfo/dev-l</a><br><br>
</blockquote>
      </div>
      <br><br clear="all"><br>
      -- <br>
      Best,<br><br>
      Arunesh Mathur<br>
      IV year, Undergraduate,<br>
      Department of Computer Science and Engineering,<br>
      National Institute of Technology Karnataka Surathkal,<br>
      India.<br><br><br><br>_______________________________________________
dev-l mailing list
<a class="moz-txt-link-abbreviated" href="mailto:dev-l@...">dev-l@...</a>
<a class="moz-txt-link-freetext" href="https://intern.openzim.org/mailman/listinfo/dev-l">https://intern.openzim.org/mailman/listinfo/dev-l</a>

    </blockquote>
    <br>
</div>
Arunesh Mathur | 5 Oct 08:22 2011
Picon

Re: [Wikitech-l] Phonegap and Wikimedia mobile apps

Hi Christian,
                   You can look at getDirectoryInfo(String articleName, char namespace) in ZimReader.java
This'll return null if there is the article doesn't exist, else the DirectoryEntry object. (Is this what you were looking for?)

Actually I plan on adding more features if required. Do you think there are any missing, when compared to zimlib? If yes, could you list them?

Have you tested zimreader-java on an Android phone? How is it's performance?

On Mon, Oct 3, 2011 at 11:10 PM, Christian Pühringer <cip-RbZlAiThDcE@public.gmane.org> wrote:
Hi Arunesh,

Firstly, thank you again for doing the java port.

I've one question regarding the features of the port:
Is there or do you plan to add a feature for iterating over the index? (Similar to find/findByTitle in C++ zimlib)
This would be required to add some auto complete feature to the phonegap app.

Best regards,
Christian

Am 24.09.2011 14:42, schrieb Arunesh Mathur:
Yes, images are not compressed in ZIM files, as Emmanuel pointed out.

Also, I have tried decompressing a LZMA compressed file on an Android phone (HTC Wildfire to be precise), and the decompression speed is not a problem.

LZMA-Java is under frequent development by Lasse Collin, so we should make sure that the latest code is used.

On Sat, Sep 24, 2011 at 6:00 PM, Emmanuel Engelhart <emmanuel-dU3mOXAlhucBXFe83j6qeQ@public.gmane.org> wrote:
On 24/09/2011 14:24, Christian Pühringer wrote:
> The JAVA liblzma performance is pretty bad: To increase efficiency of
> compression in the zim-format articles (and also all
> other data like images) are stored in clusters. Cluster size is apparently about
> 1 MB. This implies that loading an article
> which is stored at the end of a cluster involves decompressing the complete
> cluster.

Images should not be compressed in ZIM files for the obvious reasons
they mainly are already compressed. This is the case for all ZIM files I
made. As far as I know this is also the case for Mediawiki:Collection
build ZIM files.

Emmanuel


_______________________________________________
dev-l mailing list
dev-l <at> openzim.org
https://intern.openzim.org/mailman/listinfo/dev-l




--
Best,

Arunesh Mathur
IV year, Undergraduate,
Department of Computer Science and Engineering,
National Institute of Technology Karnataka Surathkal,
India.



_______________________________________________ dev-l mailing list dev-l-rOD9jy6iqmlAfugRpC6u6w@public.gmane.org https://intern.openzim.org/mailman/listinfo/dev-l




--
Best,

Arunesh Mathur
IV year, Undergraduate,
Department of Computer Science and Engineering,
National Institute of Technology Karnataka Surathkal,
India.

<div>
<p>Hi Christian,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; You can look at getDirectoryInfo(String articleName, char namespace) in ZimReader.java<br>This'll return null if there is the article doesn't exist, else the DirectoryEntry object. (Is this what you were looking for?)<br><br>Actually I plan on adding more features if required. Do you think there are any missing, when compared to zimlib? If yes, could you list them?<br><br>Have you tested zimreader-java on an Android phone? How is it's performance? <br><br></p>
<div class="gmail_quote">On Mon, Oct 3, 2011 at 11:10 PM, Christian P&uuml;hringer <span dir="ltr">&lt;<a href="mailto:cip@..." target="_blank">cip@...</a>&gt;</span> wrote:<br><blockquote class="gmail_quote">

    

  <div bgcolor="#FFFFFF" text="#000000">
    Hi Arunesh,<br><br>
    Firstly, thank you again for doing the java port.<br><br>
    I've one question regarding the features of the port: <br>
    Is there or do you plan to add a feature for iterating over the
    index? (Similar to find/findByTitle in C++ zimlib)<br>
    This would be required to add some auto complete feature to the
    phonegap app.<br><br>
    Best regards,<br>
    Christian<br><br>
    Am <a href="tel:24.09.2011%2014" value="+12409201114" target="_blank">24.09.2011 14</a>:42, schrieb Arunesh Mathur:
    <div>
<div></div>
<div>
<blockquote type="cite">Yes, images are not compressed in ZIM files, as
      Emmanuel pointed out.<br><br>
      Also, I have tried decompressing a LZMA compressed file on an
      Android phone (HTC Wildfire to be precise), and the decompression
      speed is not a problem. <br><br>
      LZMA-Java is under frequent development by Lasse Collin, so we
      should make sure that the latest code is used.<br><br><div class="gmail_quote">On Sat, Sep 24, 2011 at 6:00 PM, Emmanuel
        Engelhart <span dir="ltr">&lt;<a href="mailto:emmanuel <at> engelhart.org" target="_blank">emmanuel@...</a>&gt;</span>
        wrote:<br><blockquote class="gmail_quote">On
          24/09/2011 14:24, Christian P&uuml;hringer wrote:<br>
          &gt; The JAVA liblzma performance is pretty bad: To increase
          efficiency of<br>
          &gt; compression in the zim-format articles (and also all<br>
          &gt; other data like images) are stored in clusters. Cluster
          size is apparently about<br>
          &gt; 1 MB. This implies that loading an article<br>
          &gt; which is stored at the end of a cluster involves
          decompressing the complete<br>
          &gt; cluster.<br><br>
          Images should not be compressed in ZIM files for the obvious
          reasons<br>
          they mainly are already compressed. This is the case for all
          ZIM files I<br>
          made. As far as I know this is also the case for
          Mediawiki:Collection<br>
          build ZIM files.<br><br>
            Emmanuel<br><br><br>
          _______________________________________________<br>
          dev-l mailing list<br><a href="mailto:dev-l@..." target="_blank">dev-l <at> openzim.org</a><br><a href="https://intern.openzim.org/mailman/listinfo/dev-l" target="_blank">https://intern.openzim.org/mailman/listinfo/dev-l</a><br><br>
</blockquote>
      </div>
      <br><br clear="all"><br>
      -- <br>
      Best,<br><br>
      Arunesh Mathur<br>
      IV year, Undergraduate,<br>
      Department of Computer Science and Engineering,<br>
      National Institute of Technology Karnataka Surathkal,<br>
      India.<br><br><br><br>_______________________________________________
dev-l mailing list
<a href="mailto:dev-l@..." target="_blank">dev-l@...</a>
<a href="https://intern.openzim.org/mailman/listinfo/dev-l" target="_blank">https://intern.openzim.org/mailman/listinfo/dev-l</a>

    </blockquote>
    <br>
</div>
</div>
</div>

</blockquote>
</div>
<br><br clear="all"><br>-- <br>Best,<br><br>Arunesh Mathur<br>IV year, Undergraduate,<br>Department of Computer Science and Engineering,<br>National Institute of Technology Karnataka Surathkal,<br>India.<br><br>
</div>
Emmanuel Engelhart | 5 Oct 15:50 2011

Kiwix 0.9 beta3 is out!

Hi

Almost two months after the beta2, we finally released the 0.9 beta3.

This version fixes many bugs and improves the overall usability ; we
especially fixed two critical bugs impacting Windows, finished the file
association (special icons for ZIM files) on Linux and Windows.
Noticeable is also the new 50 UI supported languages thank to the big
work done by the Translatewiki community of translators.

You may find the detailed CHANGELOG here:
http://changelog.kiwix.org

At the same time, thank to the financial project of Wikimedia CH:
* A first version for Sugar (used by OLPC XO) was prepared. We still
need feedback to detect and fix last issues:
http://activities.sugarlabs.org/en-US/sugar/addon/4483
* A static version of Kiwix for GNU/Linux x86_32. We also need your
feedbacks here:
http://sourceforge.net/projects/kiwix/files/kiwix-0.9-beta3-linux.tar.bz2/download

... More information about the "Black&White" project:
http://www.kiwix.org/index.php/Black%26White_Project

Next release is planned for the middle of November and should still
include usability improvements thank to the continuous support of the
WMF. We want to finally release 0.9 before the end of the year, so this
will certainly be also the first release candidate (so no new feature
inclusion any more).

Follow us:
http://identi.ca/group/kiwix

Regards
Emmanuel

Hi

Almost two months after the beta2, we finally released the 0.9 beta3.

This version fixes many bugs and improves the overall usability ; we
especially fixed two critical bugs impacting Windows, finished the file
association (special icons for ZIM files) on Linux and Windows.
Noticeable is also the new 50 UI supported languages thank to the big
work done by the Translatewiki community of translators.

You may find the detailed CHANGELOG here:
http://changelog.kiwix.org

At the same time, thank to the financial project of Wikimedia CH:
* A first version for Sugar (used by OLPC XO) was prepared. We still
need feedback to detect and fix last issues:
http://activities.sugarlabs.org/en-US/sugar/addon/4483
* A static version of Kiwix for GNU/Linux x86_32. We also need your
feedbacks here:
http://sourceforge.net/projects/kiwix/files/kiwix-0.9-beta3-linux.tar.bz2/download

... More information about the "Black&White" project:
http://www.kiwix.org/index.php/Black%26White_Project

Next release is planned for the middle of November and should still
include usability improvements thank to the continuous support of the
WMF. We want to finally release 0.9 before the end of the year, so this
will certainly be also the first release candidate (so no new feature
inclusion any more).

Follow us:
http://identi.ca/group/kiwix

Regards
Emmanuel

Christian Pühringer | 6 Oct 23:01 2011
Picon
Picon

Re: [Wikitech-l] Phonegap and Wikimedia mobile apps

Hi Arunesh,

I'd need something like findByTitle(namespace, title)  in zimlib: It returns an iterator pointing
to the lexicographically next article.  I'd like to use this  to implement an auto suggest
feature (Same as in online wikipedia search box).
Note: While for auto suggest iterating forward is necessary, for other features it would be good if it is also
possible to iterate backward (same as in zimlib).

One other thing which may be currently missing is support for title and url search/get article
functionality. It looks to me the right now everything operates on urls and not on titles.
For searching it is necessary to also have title based functions. (Like findByTitle).
Note that while  currently in most zimfiles urls have the form  'Namespace'/'Title', this is not guaranteed,
the url could be completely different from the title.

Regarding performance, have you missed Brion's and my results posted in this thread?
(They may be only in the wikitech-l-RusutVdil2icGmH+5r0DM0B+6BGkLq7r@public.gmane.org and not in the dev-l-rOD9jy6iqmlAfugRpC6u6w@public.gmane.org list)


Best regards,
Christian


Am 05.10.2011 08:22, schrieb Arunesh Mathur:
Hi Christian,
                   You can look at getDirectoryInfo(String articleName, char namespace) in ZimReader.java
This'll return null if there is the article doesn't exist, else the DirectoryEntry object. (Is this what you were looking for?)

Actually I plan on adding more features if required. Do you think there are any missing, when compared to zimlib? If yes, could you list them?

Have you tested zimreader-java on an Android phone? How is it's performance?

On Mon, Oct 3, 2011 at 11:10 PM, Christian Pühringer <cip-RbZlAiThDcE@public.gmane.org> wrote:
Hi Arunesh,

Firstly, thank you again for doing the java port.

I've one question regarding the features of the port:
Is there or do you plan to add a feature for iterating over the index? (Similar to find/findByTitle in C++ zimlib)
This would be required to add some auto complete feature to the phonegap app.

Best regards,
Christian

Am 24.09.2011 14:42, schrieb Arunesh Mathur:
Yes, images are not compressed in ZIM files, as Emmanuel pointed out.

Also, I have tried decompressing a LZMA compressed file on an Android phone (HTC Wildfire to be precise), and the decompression speed is not a problem.

LZMA-Java is under frequent development by Lasse Collin, so we should make sure that the latest code is used.

On Sat, Sep 24, 2011 at 6:00 PM, Emmanuel Engelhart <emmanuel-dU3mOXAlhucBXFe83j6qeQ@public.gmane.org> wrote:
On 24/09/2011 14:24, Christian Pühringer wrote:
> The JAVA liblzma performance is pretty bad: To increase efficiency of
> compression in the zim-format articles (and also all
> other data like images) are stored in clusters. Cluster size is apparently about
> 1 MB. This implies that loading an article
> which is stored at the end of a cluster involves decompressing the complete
> cluster.

Images should not be compressed in ZIM files for the obvious reasons
they mainly are already compressed. This is the case for all ZIM files I
made. As far as I know this is also the case for Mediawiki:Collection
build ZIM files.

Emmanuel


_______________________________________________
dev-l mailing list
dev-l-rOD9jy6iqmlAfugRpC6u6w@public.gmane.org
https://intern.openzim.org/mailman/listinfo/dev-l




--
Best,

Arunesh Mathur
IV year, Undergraduate,
Department of Computer Science and Engineering,
National Institute of Technology Karnataka Surathkal,
India.



_______________________________________________ dev-l mailing list dev-l-rOD9jy6iqmlAfugRpC6u6w@public.gmane.org https://intern.openzim.org/mailman/listinfo/dev-l




--
Best,

Arunesh Mathur
IV year, Undergraduate,
Department of Computer Science and Engineering,
National Institute of Technology Karnataka Surathkal,
India.


<div>
    Hi Arunesh,<br><br>
    I'd need something like findByTitle(namespace, title)&nbsp; in zimlib: It
    returns an iterator pointing<br>
    to the lexicographically next article.&nbsp; I'd like to use this&nbsp; to
    implement an auto suggest<br>
    feature (Same as in online wikipedia search box).<br>
    Note: While for auto suggest iterating forward is necessary, for
    other features it would be good if it is also <br>
    possible to iterate backward (same as in zimlib).<br><br>
    One other thing which may be currently missing is support for title
    and url search/get article<br>
    functionality. It looks to me the right now everything operates on
    urls and not on titles.<br>
    For searching it is necessary to also have title based functions.
    (Like findByTitle).<br>
    Note that while&nbsp; currently in most zimfiles urls have the form&nbsp;
    'Namespace'/'Title', this is not guaranteed,<br>
    the url could be completely different from the title.<br><br>
    Regarding performance, have you missed Brion's and my results posted
    in this thread? <br>
    (They may be only in the <a class="moz-txt-link-abbreviated" href="mailto:wikitech-l@...">wikitech-l@...</a> and not in
    the <a class="moz-txt-link-abbreviated" href="mailto:dev-l@...">dev-l@...</a> list)<br><br><br>
    Best regards,<br>
    Christian<br><br><br>
    Am 05.10.2011 08:22, schrieb Arunesh Mathur:
    <blockquote cite="mid:CAF62jJDCAg-igrMwbCbbvye55pwZzJYaSgwusDZc3T7b5WrC3g@..." type="cite">Hi Christian,<br>
      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; You can look at getDirectoryInfo(String
      articleName, char namespace) in ZimReader.java<br>
      This'll return null if there is the article doesn't exist, else
      the DirectoryEntry object. (Is this what you were looking for?)<br><br>
      Actually I plan on adding more features if required. Do you think
      there are any missing, when compared to zimlib? If yes, could you
      list them?<br><br>
      Have you tested zimreader-java on an Android phone? How is it's
      performance? <br><br><div class="gmail_quote">On Mon, Oct 3, 2011 at 11:10 PM,
        Christian P&uuml;hringer <span dir="ltr">&lt;<a moz-do-not-send="true" href="mailto:cip@..." target="_blank">cip@...</a>&gt;</span> wrote:<br><blockquote class="gmail_quote">
          <div bgcolor="#FFFFFF" text="#000000"> Hi Arunesh,<br><br>
            Firstly, thank you again for doing the java port.<br><br>
            I've one question regarding the features of the port: <br>
            Is there or do you plan to add a feature for iterating over
            the index? (Similar to find/findByTitle in C++ zimlib)<br>
            This would be required to add some auto complete feature to
            the phonegap app.<br><br>
            Best regards,<br>
            Christian<br><br>
            Am <a moz-do-not-send="true" href="tel:24.09.2011%2014" value="+12409201114" target="_blank">24.09.2011 14</a>:42,
            schrieb Arunesh Mathur:
            <div>
              <div>
                <blockquote type="cite">Yes, images are not compressed
                  in ZIM files, as Emmanuel pointed out.<br><br>
                  Also, I have tried decompressing a LZMA compressed
                  file on an Android phone (HTC Wildfire to be precise),
                  and the decompression speed is not a problem. <br><br>
                  LZMA-Java is under frequent development by Lasse
                  Collin, so we should make sure that the latest code is
                  used.<br><br><div class="gmail_quote">On Sat, Sep 24, 2011 at 6:00
                    PM, Emmanuel Engelhart <span dir="ltr">&lt;<a moz-do-not-send="true" href="mailto:emmanuel@..." target="_blank">emmanuel@...</a>&gt;</span>
                    wrote:<br><blockquote class="gmail_quote">On

                      24/09/2011 14:24, Christian P&uuml;hringer wrote:<br>
                      &gt; The JAVA liblzma performance is pretty bad:
                      To increase efficiency of<br>
                      &gt; compression in the zim-format articles (and
                      also all<br>
                      &gt; other data like images) are stored in
                      clusters. Cluster size is apparently about<br>
                      &gt; 1 MB. This implies that loading an article<br>
                      &gt; which is stored at the end of a cluster
                      involves decompressing the complete<br>
                      &gt; cluster.<br><br>
                      Images should not be compressed in ZIM files for
                      the obvious reasons<br>
                      they mainly are already compressed. This is the
                      case for all ZIM files I<br>
                      made. As far as I know this is also the case for
                      Mediawiki:Collection<br>
                      build ZIM files.<br><br>
                        Emmanuel<br><br><br>
                      _______________________________________________<br>
                      dev-l mailing list<br><a moz-do-not-send="true" href="mailto:dev-l@..." target="_blank">dev-l@...</a><br><a moz-do-not-send="true" href="https://intern.openzim.org/mailman/listinfo/dev-l" target="_blank">https://intern.openzim.org/mailman/listinfo/dev-l</a><br><br>
</blockquote>
                  </div>
                  <br><br clear="all"><br>
                  -- <br>
                  Best,<br><br>
                  Arunesh Mathur<br>
                  IV year, Undergraduate,<br>
                  Department of Computer Science and Engineering,<br>
                  National Institute of Technology Karnataka Surathkal,<br>
                  India.<br><br><br><br>_______________________________________________
dev-l mailing list
<a moz-do-not-send="true" href="mailto:dev-l@..." target="_blank">dev-l@...</a>
<a moz-do-not-send="true" href="https://intern.openzim.org/mailman/listinfo/dev-l" target="_blank">https://intern.openzim.org/mailman/listinfo/dev-l</a>

                </blockquote>
                <br>
</div>
            </div>
          </div>
        </blockquote>
      </div>
      <br><br clear="all"><br>
      -- <br>
      Best,<br><br>
      Arunesh Mathur<br>
      IV year, Undergraduate,<br>
      Department of Computer Science and Engineering,<br>
      National Institute of Technology Karnataka Surathkal,<br>
      India.<br><br>
</blockquote>
    <br>
</div>
Arunesh Mathur | 7 Oct 12:39 2011
Picon

Re: [Wikitech-l] Phonegap and Wikimedia mobile apps

Hi Christian,
                   Sure, that seems easy enough to implement. I'll get that feature up and running soon.

Yes, most of the functions are based on title, I did that because I didn't find any difference between the two when I browsed around 5-6 ZIM files. But as you say, I'll add their URL counterparts as well. I'll also add the documentation-gen module to the repository.

Hmm, I'm not subscribed to that mailing list. I'll go through the thread that discusses performance.

-Arunesh Mathur

On Fri, Oct 7, 2011 at 2:31 AM, Christian Pühringer <cip-RbZlAiThDcE@public.gmane.org> wrote:
Hi Arunesh,

I'd need something like findByTitle(namespace, title)  in zimlib: It returns an iterator pointing
to the lexicographically next article.  I'd like to use this  to implement an auto suggest
feature (Same as in online wikipedia search box).
Note: While for auto suggest iterating forward is necessary, for other features it would be good if it is also
possible to iterate backward (same as in zimlib).

One other thing which may be currently missing is support for title and url search/get article
functionality. It looks to me the right now everything operates on urls and not on titles.
For searching it is necessary to also have title based functions. (Like findByTitle).
Note that while  currently in most zimfiles urls have the form  'Namespace'/'Title', this is not guaranteed,
the url could be completely different from the title.

Regarding performance, have you missed Brion's and my results posted in this thread?
(They may be only in the wikitech-l-RusutVdil2icGmH+5r0DM0B+6BGkLq7r@public.gmane.org and not in the dev-l <at> openzim.org list)


Best regards,
Christian


Am 05.10.2011 08:22, schrieb Arunesh Mathur:
Hi Christian,
                   You can look at getDirectoryInfo(String articleName, char namespace) in ZimReader.java
This'll return null if there is the article doesn't exist, else the DirectoryEntry object. (Is this what you were looking for?)

Actually I plan on adding more features if required. Do you think there are any missing, when compared to zimlib? If yes, could you list them?

Have you tested zimreader-java on an Android phone? How is it's performance?

On Mon, Oct 3, 2011 at 11:10 PM, Christian Pühringer <cip-RbZlAiThDcE@public.gmane.org> wrote:
Hi Arunesh,

Firstly, thank you again for doing the java port.

I've one question regarding the features of the port:
Is there or do you plan to add a feature for iterating over the index? (Similar to find/findByTitle in C++ zimlib)
This would be required to add some auto complete feature to the phonegap app.

Best regards,
Christian

Am 24.09.2011 14:42, schrieb Arunesh Mathur:
Yes, images are not compressed in ZIM files, as Emmanuel pointed out.

Also, I have tried decompressing a LZMA compressed file on an Android phone (HTC Wildfire to be precise), and the decompression speed is not a problem.

LZMA-Java is under frequent development by Lasse Collin, so we should make sure that the latest code is used.

On Sat, Sep 24, 2011 at 6:00 PM, Emmanuel Engelhart <emmanuel-dU3mOXAlhucBXFe83j6qeQ@public.gmane.org> wrote:
On 24/09/2011 14:24, Christian Pühringer wrote:
> The JAVA liblzma performance is pretty bad: To increase efficiency of
> compression in the zim-format articles (and also all
> other data like images) are stored in clusters. Cluster size is apparently about
> 1 MB. This implies that loading an article
> which is stored at the end of a cluster involves decompressing the complete
> cluster.

Images should not be compressed in ZIM files for the obvious reasons
they mainly are already compressed. This is the case for all ZIM files I
made. As far as I know this is also the case for Mediawiki:Collection
build ZIM files.

Emmanuel


_______________________________________________
dev-l mailing list
dev-l-rOD9jy6iqmlAfugRpC6u6w@public.gmane.org
https://intern.openzim.org/mailman/listinfo/dev-l




--
Best,

Arunesh Mathur
IV year, Undergraduate,
Department of Computer Science and Engineering,
National Institute of Technology Karnataka Surathkal,
India.



_______________________________________________ dev-l mailing list dev-l-rOD9jy6iqmlAfugRpC6u6w@public.gmane.org https://intern.openzim.org/mailman/listinfo/dev-l




--
Best,

Arunesh Mathur
IV year, Undergraduate,
Department of Computer Science and Engineering,
National Institute of Technology Karnataka Surathkal,
India.





.

<div>
<p>Hi Christian, <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Sure, that seems easy enough to implement. I'll get that feature up and running soon. <br><br>Yes, most of the functions are based on title, I did that because I didn't find any difference between the two when I browsed around 5-6 ZIM files. But as you say, I'll add their URL counterparts as well. I'll also add the documentation-gen module to the repository.<br><br>Hmm, I'm not subscribed to that mailing list. I'll go through the thread that discusses performance.<br><br>-Arunesh Mathur<br><br></p>
<div class="gmail_quote">On Fri, Oct 7, 2011 at 2:31 AM, Christian P&uuml;hringer <span dir="ltr">&lt;<a href="mailto:cip@...">cip@...</a>&gt;</span> wrote:<br><blockquote class="gmail_quote">

    

  <div bgcolor="#FFFFFF" text="#000000">
    Hi Arunesh,<br><br>
    I'd need something like findByTitle(namespace, title)&nbsp; in zimlib: It
    returns an iterator pointing<br>
    to the lexicographically next article.&nbsp; I'd like to use this&nbsp; to
    implement an auto suggest<br>
    feature (Same as in online wikipedia search box).<br>
    Note: While for auto suggest iterating forward is necessary, for
    other features it would be good if it is also <br>
    possible to iterate backward (same as in zimlib).<br><br>
    One other thing which may be currently missing is support for title
    and url search/get article<br>
    functionality. It looks to me the right now everything operates on
    urls and not on titles.<br>
    For searching it is necessary to also have title based functions.
    (Like findByTitle).<br>
    Note that while&nbsp; currently in most zimfiles urls have the form&nbsp;
    'Namespace'/'Title', this is not guaranteed,<br>
    the url could be completely different from the title.<br><br>
    Regarding performance, have you missed Brion's and my results posted
    in this thread? <br>
    (They may be only in the <a href="mailto:wikitech-l@...rg" target="_blank">wikitech-l@...</a> and not in
    the <a href="mailto:dev-l@..." target="_blank">dev-l <at> openzim.org</a> list)<br><br><br>
    Best regards,<br>
    Christian<br><br><br>
    Am 05.10.2011 08:22, schrieb Arunesh Mathur:
    <div>
<div></div>
<div class="h5">
<blockquote type="cite">Hi Christian,<br>
      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; You can look at getDirectoryInfo(String
      articleName, char namespace) in ZimReader.java<br>
      This'll return null if there is the article doesn't exist, else
      the DirectoryEntry object. (Is this what you were looking for?)<br><br>
      Actually I plan on adding more features if required. Do you think
      there are any missing, when compared to zimlib? If yes, could you
      list them?<br><br>
      Have you tested zimreader-java on an Android phone? How is it's
      performance? <br><br><div class="gmail_quote">On Mon, Oct 3, 2011 at 11:10 PM,
        Christian P&uuml;hringer <span dir="ltr">&lt;<a href="mailto:cip <at> gmx.at" target="_blank">cip@...</a>&gt;</span> wrote:<br><blockquote class="gmail_quote">
          <div bgcolor="#FFFFFF" text="#000000"> Hi Arunesh,<br><br>
            Firstly, thank you again for doing the java port.<br><br>
            I've one question regarding the features of the port: <br>
            Is there or do you plan to add a feature for iterating over
            the index? (Similar to find/findByTitle in C++ zimlib)<br>
            This would be required to add some auto complete feature to
            the phonegap app.<br><br>
            Best regards,<br>
            Christian<br><br>
            Am <a href="tel:24.09.2011%2014" value="+12409201114" target="_blank">24.09.2011 14</a>:42,
            schrieb Arunesh Mathur:
            <div>
              <div>
                <blockquote type="cite">Yes, images are not compressed
                  in ZIM files, as Emmanuel pointed out.<br><br>
                  Also, I have tried decompressing a LZMA compressed
                  file on an Android phone (HTC Wildfire to be precise),
                  and the decompression speed is not a problem. <br><br>
                  LZMA-Java is under frequent development by Lasse
                  Collin, so we should make sure that the latest code is
                  used.<br><br><div class="gmail_quote">On Sat, Sep 24, 2011 at 6:00
                    PM, Emmanuel Engelhart <span dir="ltr">&lt;<a href="mailto:emmanuel@..." target="_blank">emmanuel@...</a>&gt;</span>
                    wrote:<br><blockquote class="gmail_quote">On

                      24/09/2011 14:24, Christian P&uuml;hringer wrote:<br>
                      &gt; The JAVA liblzma performance is pretty bad:
                      To increase efficiency of<br>
                      &gt; compression in the zim-format articles (and
                      also all<br>
                      &gt; other data like images) are stored in
                      clusters. Cluster size is apparently about<br>
                      &gt; 1 MB. This implies that loading an article<br>
                      &gt; which is stored at the end of a cluster
                      involves decompressing the complete<br>
                      &gt; cluster.<br><br>
                      Images should not be compressed in ZIM files for
                      the obvious reasons<br>
                      they mainly are already compressed. This is the
                      case for all ZIM files I<br>
                      made. As far as I know this is also the case for
                      Mediawiki:Collection<br>
                      build ZIM files.<br><br>
                        Emmanuel<br><br><br>
                      _______________________________________________<br>
                      dev-l mailing list<br><a href="mailto:dev-l@..." target="_blank">dev-l@...</a><br><a href="https://intern.openzim.org/mailman/listinfo/dev-l" target="_blank">https://intern.openzim.org/mailman/listinfo/dev-l</a><br><br>
</blockquote>
                  </div>
                  <br><br clear="all"><br>
                  -- <br>
                  Best,<br><br>
                  Arunesh Mathur<br>
                  IV year, Undergraduate,<br>
                  Department of Computer Science and Engineering,<br>
                  National Institute of Technology Karnataka Surathkal,<br>
                  India.<br><br><br><br>_______________________________________________
dev-l mailing list
<a href="mailto:dev-l@..." target="_blank">dev-l@...</a>
<a href="https://intern.openzim.org/mailman/listinfo/dev-l" target="_blank">https://intern.openzim.org/mailman/listinfo/dev-l</a>

                </blockquote>
                <br>
</div>
            </div>
          </div>
        </blockquote>
      </div>
      <br><br clear="all"><br>
      -- <br>
      Best,<br><br>
      Arunesh Mathur<br>
      IV year, Undergraduate,<br>
      Department of Computer Science and Engineering,<br>
      National Institute of Technology Karnataka Surathkal,<br>
      India.<br><br>
</blockquote>
    <br>
</div>
</div>
</div>

</blockquote>
</div>
<br><br clear="all"><br>.<br><br>
</div>

Gmane