Gary Russo | 16 Sep 22:56 2014
Picon

Re: [MarkLogic Dev General] Leading wildcard search returning wrong total

Thanks Mike!

As you noted, using the recommended configuration posted on this page did
the trick. => https://docs.marklogic.com/guide/search-dev/wildcard#id_14163

Key settings used:
 - word searches
 - three character word searches
 - word positions
 - word lexicon in the codepoint collation
 - three character word positions

It also simplifies the $options.

This is the query that I use now.

import module namespace search = "http://marklogic.com/appservices/search"
    at "/MarkLogic/appservices/search/search.xqy";

declare variable $DN := "http://demo.org/demo";

let $options :=
  <options xmlns="http://marklogic.com/appservices/search">
    <constraint name="ImportedUnitCode">
      <word>
        <element ns="http://demo.org/demo" name="ImportedUnitCode"/>
      </word>
    </constraint>
    <sort-order type="xs:string" collation="http://marklogic.com/collation/"
direction="ascending">
(Continue reading)

Tim | 15 Sep 18:15 2014
Picon

[MarkLogic Dev General] How to access XML config files from a modules database instead of content database?

Hi Folks,

I'm wondering how to access XML configuration files from a modules database
while the code is being executed in the context of a content database? Is there
a specific import command that allows me to do this?

Tim M.

Tim | 10 Sep 18:27 2014
Picon

[MarkLogic Dev General] How to export XML to ASCII text with ISO encodings?

Hi Folks,

 

I am importing content from an ASCII text file into an XML record to manage updates in MarkLogic, but upon request I need to export an updated ASCII text file. The ASCII text file can include a variety of ISO encodings and upon export it is required that some ASCII characters (such as apostrophes and ampersands) be out put with their equivalent ISO encoding, i.e., “&apos;” and “&amp;”) and that any characters exceeding the ASCII range be output with ISO encodings, e.g., “&uuml;”.

 

First of all I need to ensure that any text content with ISO encodings can be adequately imported into its UTF-8 representation.  Secondly, upon export to a text file, I need to add back in the ISO entity encodings. Any suggestions?

 

Thank you!

 

Tim M.

 

<div><div class="WordSection1">
<p class="MsoNormal"><span>Hi Folks,<p></p></span></p>
<p class="MsoNormal"><span><p>&nbsp;</p></span></p>
<p class="MsoNormal"><span>I am importing content from an ASCII text file into an XML record to manage updates in MarkLogic, but upon request I need to export an updated ASCII text file. The ASCII text file can include a variety of ISO encodings and upon export it is required that some ASCII characters (such as apostrophes and ampersands) be out put with their equivalent ISO encoding, i.e., &ldquo;&amp;apos;&rdquo; and &ldquo;&amp;amp;&rdquo;) and that any characters exceeding the ASCII range be output with ISO encodings, e.g., &ldquo;&amp;uuml;&rdquo;.<p></p></span></p>
<p class="MsoNormal"><span><p>&nbsp;</p></span></p>
<p class="MsoNormal"><span>First of all I need to ensure that any text content with ISO encodings can be adequately imported into its UTF-8 representation.&nbsp; Secondly, upon export to a text file, I need to add back in the ISO entity encodings. Any suggestions?<p></p></span></p>
<p class="MsoNormal"><span><p>&nbsp;</p></span></p>
<p class="MsoNormal"><span>Thank you!<p></p></span></p>
<p class="MsoNormal"><span><p>&nbsp;</p></span></p>
<p class="MsoNormal"><span>Tim M.<p></p></span></p>
<p class="MsoNormal"><span><p>&nbsp;</p></span></p>
</div></div>
priya dharshni | 10 Sep 10:42 2014
Picon

[MarkLogic Dev General] Identify the uris which does not have particular element.


Hi ,


  I want to identify the uris which does not have the particular element.Say, i want to get uris which does not have "id' element.
 
 xml in db :
  <employee>
  <name>test</name>
  <id>123</id>
  </employee>
 
  cts:uris(''",
                          (),
               
                              
                          cts:not-query(cts:element-value-query(xs:QName("id"),"*"))
            )
The above query pulls all the uris with and without "id" element.Please let me know how to fix this.

Regards,
Priya.
<div><div dir="ltr">
<br>Hi ,<br><br><br>&nbsp; I want to identify the uris which does not have the particular element.Say, i want to get uris which does not have "id' element.<br>&nbsp;<br>&nbsp;xml in db :<br>&nbsp; &lt;employee&gt;<br>&nbsp; &lt;name&gt;test&lt;/name&gt;<br>&nbsp; &lt;id&gt;123&lt;/id&gt;<br>&nbsp; &lt;/employee&gt;<br>&nbsp; <br>&nbsp; cts:uris(''",<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; (), <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; cts:not-query(cts:element-value-query(xs:QName("id"),"*"))<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ) <br>The above query pulls all the uris with and without "id" element.Please let me know how to fix this.<br><br>Regards,<br>Priya.<br>
</div></div>
Florent Georges | 8 Sep 13:31 2014

[MarkLogic Dev General] Virtual box to install MarkLogic for dev

  Hi,

  I wrote a blog post last weekend about creating a virtual box for
installing MarkLogic.  This is only for development purposes, in order
to help installing a new version of MarkLogic when you already got one
in your system.  I thought it might be of interst here:

    http://h2oconsulting.be/blog/virtual-box-marklogic

  Regards,
 
--
Florent Georges
http://fgeorges.org/
http://h2oconsulting.be/


<div><div>
<div class=""><span class="">&nbsp; Hi,<br><br class="">&nbsp; I wrote a blog post last weekend about creating a virtual box for<br class="">installing MarkLogic.&nbsp; This is only for development purposes, in order<br class="">to help installing a new version of MarkLogic when you already got one<br class="">in your system.&nbsp; I thought it might be of interst here:<br class=""><br class="">&nbsp;&nbsp;&nbsp; http://h2oconsulting.be/blog/virtual-box-marklogic<br class=""><br class="">&nbsp; Regards,<br class=""></span></div>
<div class="">&nbsp;</div>
<div class="">-- <br class="">Florent Georges<br class="">http://fgeorges.org/<br class="">http://h2oconsulting.be/<br><br><br>
</div>
</div></div>
"neil bradley" | 8 Sep 12:22 2014
Picon

[MarkLogic Dev General] xsl:analyze-string grouping bug?

Hi,

I think there is a bug in the MarkLogic implementation of the
<xsl:analyze-string> element.

It seems that you cannot have more than one character between regex
groups.

My original issue was date ranges, where there could be a “-“ or “- “
between the dates, and wasted some some investigating hyphens in
character classes.

But the problem occurs with ANY two characters, even if I know the
exact order. So, if the dates were going to be formatted like this:

  <Dates>2004xy2006</Dates>

Then it still fails when I try this:

<xsl:template match="Dates">
  <xsl:variable name="inputString"><xsl:value-of
select="."/></xsl:variable>
  <xsl:choose>
    <xsl:when test="matches($inputString, '(\d\d\d\d)xy(\d\d\d\d)',
'i')">
      <xsl:analyze-string select="$inputString"
regex="(\d\d\d\d)xy(\d\d\d\d)" flags="i">
        <xsl:matching-substring>
          <Dates>
            <Date><xsl:value-of select="regex-group(1)"/></Date>
            <Date><xsl:value-of select="regex-group(2)"/></Date>
          </Dates>
        </xsl:matching-substring>
      </xsl:analyze-string>
    </xsl:when>
  </xsl:choose>
</xsl:template>

The matches() function works fine, but the second <Date> element in
the output is empty. So the  regex attribute is not working correctly.

The problem cannot be avoided by using a character class [...] *,  or
even by adding + or* to a single character!

Incidentally, it does recover thereafter. So, if there is a third
group, then that will be output correctly.

Of course, the solution is to just make the intermediate characters
into a group too, even though I don’t want that group. But I still
think this is a bug that is worth noting.

Neil.
_______________________________________________
General mailing list
General <at> developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
Mrinmoy Khamrui | 5 Sep 15:59 2014

[MarkLogic Dev General] using cts:boost-query with search:resolve in Marklogic 7

Hi Folks,

I am trying to use cts:boost-query with search:resolve and getting invalid coercion. Below is my code snippet.

let $query := search:parse($search-term,$options)
let $query := cts:boost-query(cts:query($query), cts:path-range-query("somepath with path index", ">", 0,
  ("score-function=linear","slope-factor=10")))

let $results := search:resolve($query, $options, $start, $pageRecs)

Any suggestion is highly appreciated.

Thanks
Mrinmoy
<div><div dir="ltr">Hi Folks,<div><br></div>
<div>I am trying to use cts:boost-query with search:resolve and getting invalid coercion. Below is my code snippet.</div>
<div><br></div>
<div>let $query :=&nbsp;search:parse($search-term,$options)</div>
<div>
<div>let $query := cts:boost-query(cts:query($query), cts:path-range-query("somepath with path index", "&gt;", 0,</div>
<div>&nbsp; ("score-function=linear","slope-factor=10")))</div>
</div>
<div><br></div>
<div>let $results := search:resolve($query, $options, $start, $pageRecs)<br>
</div>
<div><br></div>
<div>Any suggestion is highly appreciated.</div>
<div><br></div>
<div>Thanks</div>
<div>Mrinmoy</div>
</div></div>
Danny Sinang | 5 Sep 00:47 2014
Picon

[MarkLogic Dev General] Number of fragments per document

Hi,

I noticed there are more fragments than documents in my ML databases which have no Fragment Roots or Fragment Parents configured.

Is this by design ?

Are document properties stored in another fragment ? If so, then there should be twice the number of fragments than documents. But how come I see 2.6x fragments than my documents ?


Regards,
Danny


<div><div dir="ltr">Hi,<div><br></div>
<div>I noticed there are more fragments than documents in my ML databases which have no Fragment Roots or Fragment Parents configured.</div>
<div><br></div>
<div>Is this by design ?</div>
<div><br></div>
<div>Are document properties stored in another fragment ? If so, then there should be twice the number of fragments than documents. But how come I see 2.6x fragments than my documents ?</div>
<div><br></div>
<div><br></div>
<div>Regards,</div>
<div>Danny</div>
<div><br></div>
<div><br></div>
</div></div>
Norman Walsh | 4 Sep 15:21 2014

Re: [MarkLogic Dev General] Migrating packages from ML6 to 7

"Retter, Adam (RBI-UK)" <Adam.Retter@...> writes:
> Previously with ML6 we were HTTP POST’ing a package to configure a
> clean install of an ML instance. The package API has changed in ML7
> and we are having a hard time understanding exactly how it works and
> which end-point to use.

Hi Adam,

Sorry it appears to have become more complicated. Supporting new features
made a single XML document impractical.

However, the old package format is still supported. If you've got a 6.x
package named "package.xml" in hand, you can install it in three steps:

1. Create the package "newpack". The name is irrelevant.

curl -s -X POST --digest -u admin:admin --data-ascii  <at> /dev/null \
    -H "Accept: application/xml" -H "Content-Type: application/xml" \
    "http://localhost:8002/manage/v2/packages?pkgname=newpack&format=xml"

2. Add the contents of your MarkLogic 6 package to it:

curl -s -X POST --digest -u admin:admin -d  <at> package.xml \
     -H "Accept: application/xml" -H "Content-Type: application/xml"
     "http://localhost:8002/manage/v2/packages/newpack"

3. Install it:

curl -s --data-binary  <at> /dev/null --digest -u admin:admin \
     -H "Accept: application/xml"
     "http://localhost:8002/manage/v2/packages/newpack/install?format=xml"

Hope that helps. And sorry again about the confusion.

                                        Be seeing you,
                                          norm

-- 
Norman Walsh
Lead Engineer
MarkLogic Corporation
Phone: +1 512 761 6676
www.marklogic.com
"Retter, Adam (RBI-UK)" <Adam.Retter@...> writes:
> Previously with ML6 we were HTTP POST’ing a package to configure a
> clean install of an ML instance. The package API has changed in ML7
> and we are having a hard time understanding exactly how it works and
> which end-point to use.

Hi Adam,

Sorry it appears to have become more complicated. Supporting new features
made a single XML document impractical.

However, the old package format is still supported. If you've got a 6.x
package named "package.xml" in hand, you can install it in three steps:

1. Create the package "newpack". The name is irrelevant.

curl -s -X POST --digest -u admin:admin --data-ascii  <at> /dev/null \
    -H "Accept: application/xml" -H "Content-Type: application/xml" \
    "http://localhost:8002/manage/v2/packages?pkgname=newpack&format=xml"

2. Add the contents of your MarkLogic 6 package to it:

curl -s -X POST --digest -u admin:admin -d  <at> package.xml \
     -H "Accept: application/xml" -H "Content-Type: application/xml"
     "http://localhost:8002/manage/v2/packages/newpack"

3. Install it:

curl -s --data-binary  <at> /dev/null --digest -u admin:admin \
     -H "Accept: application/xml"
     "http://localhost:8002/manage/v2/packages/newpack/install?format=xml"

Hope that helps. And sorry again about the confusion.

                                        Be seeing you,
                                          norm

--

-- 
Norman Walsh
Lead Engineer
MarkLogic Corporation
Phone: +1 512 761 6676
www.marklogic.com
Whitby, Rob | 4 Sep 13:48 2014

Re: [MarkLogic Dev General] Migrating packages from ML6 to 7

I've battled with the packaging api as well, it does seem over complicated. I think the key is upload and
install are 2 separate steps. Here's a bash script I use:

#!/usr/bin/env bash
set -e

# defaults
HOST="ml.local.springer.com"
AUTH="admin:admin"
PACKAGE_ZIP="package.zip"

function usage() {
  printf '
usage: deploy.sh [options...]
Options:
  -t <target host>       Target hostname. Defaults to ml.local.springer.com
  -c <user:password>     Credentials. Defaults to admin:admin
  -p <package zip path>  Path to package zip. Defaults to package.zip
'
  exit 1
}

while getopts 't:c:p:' OPTION
do
  case $OPTION in
    t) HOST="$OPTARG";;
    c) AUTH="$OPTARG";;
    p) PACKAGE_ZIP="$OPTARG";;
    *) usage;;
  esac
done

PACKAGE_NAME="mldeploy"
PACKAGE_LOG="/tmp/mldeploy.log"
CREDENTIALS="--digest -u ${AUTH}"
URL="http://${HOST}:8002/manage/v2/packages"

function errorcheck() {
    if [ ${PIPESTATUS[0]} != 0 ]; then
      exit 1
    fi
    grep -q error $PACKAGE_LOG && exit 1 || true
}

echo "" > $PACKAGE_LOG

echo Deploying ${PACKAGE_ZIP} as ${PACKAGE_NAME} to ${URL}

echo Deleting existing package..
curl --progress-bar -X DELETE ${CREDENTIALS} \
    "${URL}/${PACKAGE_NAME}" \
    2>&1 | tee -a "${PACKAGE_LOG}"
errorcheck

echo Uploading package..
curl --progress-bar -X POST ${CREDENTIALS} -H "Content-type: application/zip" \
    --data-binary  <at> "${PACKAGE_ZIP}" \
    "${URL}?pkgname=${PACKAGE_NAME}" \
    2>&1 | tee -a "${PACKAGE_LOG}"
errorcheck

echo Installing package..
curl --progress-bar -X POST ${CREDENTIALS} \
    --data-binary  <at> /dev/null \
    "${URL}/${PACKAGE_NAME}/install" \
    2>&1 | tee -a "${PACKAGE_LOG}"
errorcheck

echo Finished

________________________________________
From: general-bounces@...
[general-bounces@...rklogic.com] on behalf of Retter, Adam
(RBI-UK) [Adam.Retter@...]
Sent: 04 September 2014 10:51
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] Migrating packages from ML6 to 7

Previously with ML6 we were HTTP POST’ing a package to configure a clean install of an ML instance. The
package API has changed in ML7 and we are having a hard time understanding exactly how it works and which
end-point to use.

We previously posted a single XML document in ML6 containing all of our config to
http://localhost:8002/manage/v1/package/install. Studying the documentation here: https://docs.marklogic.com/guide/relnotes/chap4#id_89025

Implies that to migrate we now have to:

1)       switch to a ZIP file format of multiple XML documents and update the package config files to contain the
extra stuff in ML7. We have done that no problem.

2)      Instead now HTTP POST our ZIP file to http://localhost:8002/manage/v2/packages/{pkgname}/install<http://localhost:8002/manage/v2/packages/%7bpkgname%7d/install>

We are having problems understanding (2). We do not know what the value of `pkgname` should be, so we have
chosen an arbitrary name e.g. `devbox`. However when we perform the HTTP POST with cURL, we get the
following ` PKG-DOESNOTEXIST ` error:

[rettera <at> qhsw24691-ml7 ~]$ curl -v -X POST --digest -u admin:admin -H "Content-Type: application/zip"
--data-binary  <at> config/marklogic/devbox/devbox-package-v2.zip http://localhost:8002/manage/v2/packages/devbox/install
Warning: Couldn't read data from file
Warning: "config/marklogic/devbox/devbox-package-v2.zip", this makes an empty
Warning: POST.
* About to connect() to localhost port 8002 (#0)
*   Trying ::1... Connection refused
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 8002 (#0)
* Server auth using Digest with user 'admin'
> POST /manage/v2/packages/devbox/install HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3
libidn/1.18 libssh2/1.4.2
> Host: localhost:8002
> Accept: */*
> Content-Type: application/zip
> Content-Length: 0
>
< HTTP/1.1 401 Unauthorized
< WWW-Authenticate: Digest realm="public", qop="auth",
nonce="f55af58e243a74202dc40b1240578d15", opaque="50c2c74c190fae6e"
< Content-type: application/xml
< Server: MarkLogic
< Content-Length: 173
< Connection: Keep-Alive
< Keep-Alive: timeout=5
<
* Ignoring the response-body
* Connection #0 to host localhost left intact
* Issue another request to this URL: 'http://localhost:8002/manage/v2/packages/devbox/install'
* Re-using existing connection! (#0) with host localhost
* Connected to localhost (127.0.0.1) port 8002 (#0)
* Server auth using Digest with user 'admin'
> POST /manage/v2/packages/devbox/install HTTP/1.1
> Authorization: Digest username="admin", realm="public",
nonce="f55af58e243a74202dc40b1240578d15", uri="/manage/v2/packages/devbox/install",
cnonce="MDAwMjQ3", nc=00000001, qop="auth", response="1faa715f863f4420d8bf654e33668c77", opaque="50c2c74c190fae6e"
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3
libidn/1.18 libssh2/1.4.2
> Host: localhost:8002
> Accept: */*
> Content-Type: application/zip
> Content-Length: 0
>
< HTTP/1.1 404 Not Found
< Content-type: application/xml
< Server: MarkLogic
< Content-Length: 259
< Connection: Keep-Alive
< Keep-Alive: timeout=5
<
<error xmlns="http://marklogic.com/manage">
  <status-code>404</status-code>
  <status>Not Found</status>
  <message-code>PKG-DOESNOTEXIST</message-code>
  <message>PKG-DOESNOTEXIST: (pkg:ERR-DOESNOTEXIST) There is no package named: devbox</message>
</error>
* Connection #0 to host localhost left intact
* Closing connection #0

So what are we missing? The documentation for the new packaging API and migration to seems to be very vague,
unless I am missing some pages somewhere?

Cheers Adam.

DISCLAIMER

This message is intended only for the use of the person(s) ("Intended Recipient") to whom it is addressed.
It may contain information, which is privileged and confidential. Accordingly any dissemination,
distribution, copying or other use of this message or any of its content by any person other than the
Intended Recipient may constitute a breach of civil or criminal law and is strictly prohibited. If you are
not the Intended Recipient, please contact the sender as soon as possible.

Reed Business Information Limited. Registered Office: Quadrant House, The Quadrant, Sutton, Surrey,
SM2 5AS, UK.

Registered in England under Company No. 151537

Tim | 3 Sep 19:01 2014
Picon

[MarkLogic Dev General] Text mime mapping a file with an unusual file extension

Hi Folks,

 

I am loading a document into MarkLogic that has an unusual file extension which for sale of discussion I will call “.oth”.  If I rename the file with a .txt extension then I can easily tokenize it as a huge text string, but a nonstandard extension it is not treated as text.  I want to associate the file as a text file, but what I would like to know is if I can import the file, e.g., File.oth using webdav and then set the mime type in my xquery, or if the mime type needs to be preconfigured in the database.

 

Suggestions?

 

Thank you!

 

Tim M.

 

<div><div class="WordSection1"><div><div>
<p class="MsoNormal"><span>Hi Folks,<p></p></span></p>
<p class="MsoNormal"><span><p>&nbsp;</p></span></p>
<p class="MsoNormal"><span>I am loading a document into MarkLogic that has an unusual file extension which for sale of discussion I will call &ldquo;.oth&rdquo;. &nbsp;If I rename the file with a .txt extension then I can easily tokenize it as a huge text string, but a nonstandard extension it is not treated as text.&nbsp; I want to associate the file as a text file, but what I would like to know is if I can import the file, e.g., File.oth using webdav and then set the mime type in my xquery, or if the mime type needs to be preconfigured in the database.<p></p></span></p>
<p class="MsoNormal"><span><p>&nbsp;</p></span></p>
<p class="MsoNormal"><span>Suggestions?<p></p></span></p>
<p class="MsoNormal"><span><p>&nbsp;</p></span></p>
<p class="MsoNormal"><span>Thank you!<p></p></span></p>
<p class="MsoNormal"><span><p>&nbsp;</p></span></p>
<p class="MsoNormal"><span>Tim M.<p></p></span></p>
<p class="MsoNormal"><span><p>&nbsp;</p></span></p>
</div></div></div></div>

Gmane