The bibexport.sh script

(1)

The bibexport.sh script

Nicolas Markey 2009/10/11

Abstract

bibexport.shis a small shell script, relying on BibTEX, that extracts entries of one or several.bibfile(s). It will expand abbreviations and cross- references, except standard month and journal abbreviations. The output is indented as neatly as possible, yielding a readable .bib file even if the original one was not.

1 Exporting .bib files

1.1 Why and how?

BibTEX aims at allowing for the use of one single.bibfile, containing many entries, from which BibTEX extracts only the\cited ones. When sending a document to someone else, this requires either to send the whole file, or to extract some entries from the.bibfile.

BibTEX also has a mechanism for using abbreviations and cross-references.

When extracting entries of a large.bibfile, it can be interesting to develop those abbreviations, in order to get a clean, self-contained .bib file. Also, it may be useful to develop cross-references in a.bibfile, independently of any document.

bibexportcan either extract entries that are cited in a document, or all the entries of one or several .bib files. It will always develop cross-references and abreviations, except standard abbreviations for months or some journals, that are defined in standard BibTEX styles. This script uses BibTEX. This has both pros and cons:

+ it is very simple. Basicaly, the script simply calls BibTEX, and the.bstfile just outputs the name and the content of each field.

+ since it uses BibTEX, we are sure that it will handle everything ”properly”, i.e. in the same way as they will be handled when cited in a L^ATEX document;

= BibTEX has some strict limitations (especially ”no more than 78 consecutive non-space characters”) that we must be aware of. On the other hand, any such problem occuring within the script would also occur when compiling a document;

(2)

− abbreviations and cross-references will always be developped. It could be argued that this is also a positive point, but having the choice would have been better.

− Many people seem to find BibTEX’s internal language clumsy, and thus the script could be difficult to adapt to special needs. However, this isreallynot that difficult, as will be explained later on. And please, don’t hesitate to ask for improvements.

1.2 Related scripts

Several other tools exist for achieving this task:

• aux2bib, written by Ralf Treinen, relies on bib2bib, which is a CaML program for selecting some entries in one or several.bib files. It does not expand anything, but includes all the necessary definitions and entries.

• bibextract.sh, by Nelson Beebe. This script usesawkfor extracting some entries out of a.bibfile. It is said to be noncompliant with cross-references.

• subset.bst, by David Kotz. export.bstdevelops the same ideas (but I discov- ered that only later on). subset.bstdoes not handle@preamble, neither does it ”protect” standard abbreviations.

• Probalby some others...

1.3 Some examples

• extracting\cited references of a document, also including cross-references:

bibexport -o <result>.bib <file>.aux

• extracting\cited references of a document, without crossrefs, and using a special.bstfile:

bibexport -b <style>.bst -o <result>.bib <file>.aux

• export all the entries of two.bibfiles (including crossrefed entries):

bibexport -a -o <result>.bib <file1>.bib <file2>.bib

• export all the entries of two.bibfiles (without crossrefs):

bibexport -a -n -o <result>.bib <file1>.bib <file2>.bib In fact, the only difference between this and the previous one is that crossreffield will be filtered out at the end of the script.

(3)

• export all the entries of two .bibfiles, using an extra file containing cross- referenced entries (which should not be included):

bibexport -a -e <crossref>.bib -n -o <result>.bib \

<file1>.bib <file2>.bib

1.4 Exporting extra fields

By default,bibexport.shexports only ”standard” fields (those defined and used inplain.bst), as well as a few others. It is very easy to modify it in order to export other fields: it suffices to modifyexport.bstas follows:

• in theENTRYlist, add the name of the field you would like to export. Notice that ENTRY takes three space-separated lists as arguments; you must add extra fields in the first argument (actually, the last two are empty).

• in the functionentry.export.extra, add a line of the form

"myfield" myfield field.export

wheremyfieldis the name of the extra field you want to export.

Acknowledgements

I thank ´Eric Colin de Verdi`ere and Richard Mathar for suggesting several improvements or corrections.

2 The code

2.1 The shell script

2.1.1 Initialization

usage() We first define how the script should be used:

1h∗scripti

2function usage()

3{

4echo "bibexport: a tool to extract BibTeX entries out of .bib files.

5usage: $0 [-h|v] [-n] [-b bst] [-a [-e file]...] [-o file] file...

6 -a, --all export the entire .bib files

7 -b, --bst specifies the .bst style file [default: export.bst]

8 -c, --crossref include entries that are crossref’d [default: yes]

9 -e file, --extra file extra .bib files to be used (for crossrefs)

10 -n, --no-crossref don’t include crossref’d entries [default: no]

11 -o file write output to file [default: bibexport.bib]

12 -p, --preamble write a preamble at beginning of output

13 -t, --terse operate silently

14 -h, --help print this message and exit

(4)

15 -v, --version print version number and exit";

16exit 0;

17}

18h/scripti

VERSION VDATE CREF FILE EXT EXTRA EXTRABIB SPACE BST TERSE NOBANNER ARGS

We define the default value of some variables:

• $VERSION: the version number,

• $VDATE: the release date,

• $CREF: the value of-min-crossrefs,

• $FILE: the input file(s),

• $EXT: the extension (.auxor .bib) of input files,

• $EXTRA: list of possible extra.bibfiles without extension,

• $EXTRABIB: list of possible extra.bibfiles with extension,

• $SPACE: file name separator (can be , comma or empty),

• $BST: the.bst file to be used,

• $TERSE: tells BibTEX to run silently,

• $NOBANNER: don’t print the initial comment,

• $ARGS: the list of aruments passed tobibexport.sh.

19h∗scripti

20VERSION="2.20";

21VDATE="2009/10/11";

22CROSSREF="1";

23FILE="";

24EXT=".aux";

25EXTRA="";

26EXTRABIB="";

27SPACE=" ";

28BST="export";

29TERSE="";

30NOBANNER="true";

31ARGS=$@;

32h/scripti

2.1.2 Handling arguments

If no argument have been supplied, we callusage().

33h∗scripti

34if [ $# -eq 0 ]; then

35 usage;

36fi

37h/scripti

(5)

Otherwise, we enter awhile-loop for handling the whole list of arguments:

38h∗scripti

39while [ $# != 0 ]; do

40 case $1 in

41h/scripti

• -aor --all: export all the bibliography. This means that we input .bib files.

42 h∗scripti

43 -a|--all)

44 EXT=""; SPACE="";

45 shift ;;

46 h/scripti

• -bor--bst: specifies the style file. It seems that BibTEX does not like the ./style.bstsyntax, and we have to handle that case separately.

47 h∗scripti

48 -b|--bst)

49 if [ "‘dirname $2‘" = "." ]; then

50 DOLLARTWO="‘basename $2 .bst‘";

51 else

52 DOLLARTWO="‘dirname $2‘/‘basename $2 .bst‘";

53 fi

54 BST="${DOLLARTWO}";

55 shift 2;;

56 h/scripti

• -eor--extra: when we want to export all the entries of a.bibfile, we can specify an extra.bibfile that would contain entries that we don’t want to export, but that are needed,e.g. for crossrefs.

57 h∗scripti

58 -e|--extra)

59 if [ "‘dirname $2‘" = "." ]; then

60 DOLLARTWO="‘basename $2 .bib‘";

61 else

62 DOLLARTWO="‘dirname $2‘/‘basename $2 .bib‘";

63 fi

64 EXTRA="${EXTRA}${DOLLARTWO},";

65 EXTRABIB="${EXTRABIB},${DOLLARTWO}.bib";

66 shift 2;;

67 h/scripti

• -oor--output: the name of the output file.

68 h∗scripti

69 -o|--output-file)

70 if [ "‘dirname $2‘" = "." ]; then

71 DOLLARTWO="‘basename $2 .bib‘";

(6)

72 else

73 DOLLARTWO="‘dirname $2‘/‘basename $2 .bib‘";

74 fi

75 OUTPUT="${DOLLARTWO}.bib";

76 shift 2 ;;

77 h/scripti

• -cor --crossref(or others): this options means that we want crossrefs to be included. Note that for any entry, field inheritage will be performed.

78 h∗scripti

79 -c|--crossref|--crossrefs|--with-crossref|--with-crossrefs)

80 CREF="1" ;

81 shift ;;

82 h/scripti

• -nor--no-crossref: don’t include crossref’ed entries.

83 h∗scripti

84 -n|--no-crossref|--without-crossref)

85 CREF="20000" ;

86 shift ;;

87 h/scripti

• -vor--versionfor version number:

88 h∗scripti

89 -v|--version)

90 echo "This is bibexport v${VERSION} (released ${VDATE})"; exit 0;;

91 h/scripti

• -p or --preamble for inserting some informations at the beginning of the output file:

92 h∗scripti

93 -p|--preamble)

94 NOBANNER="false";

95 shift ;;

96 h/scripti

• -tor--tersefor asking BibTEX to run silently:

97 h∗scripti

98 -t|--terse)

99 TERSE=" -terse ";

100 shift ;;

101 h/scripti

• other dash-options are erroneous (except-h, but...):

102 h∗scripti

103 -*)

104 usage;;

105 h/scripti

(7)

• there should only remain file names: we add those names to the list of files.

106 h∗scripti

107 *)

108 if [ "‘dirname $1‘" = "." ]; then

109 DOLLARONE="‘basename $1 ${EXT}‘";

110 else

111 DOLLARONE="‘dirname $1‘/‘basename $1 ${EXT}‘";

112 fi

113 FILE="${FILE}${SPACE}${DOLLARONE}${EXT}";

114 if [ -z "${SPACE}" ]; then

115 SPACE=",";

116 fi;

117 shift;;

118 h/scripti

That’s all folks:

119h∗scripti

120 esac

121done

122h/scripti

2.1.3 The core of the script

We first set the name of the result and intermediary files:

123h∗scripti

124FINALFILE=${OUTPUT};

125if [ ! "${FINALFILE}" ]; then

126 FINALFILE="bibexport.bib";

127fi

128TMPFILE="bibexp.‘date +%s‘";

129h/scripti

We then create the .aux file for the main run of BibTEX. Note that this could call BibTEX, with theexpkeys.bstfile, in the case where we want to export all entries of a .bib file but not crossrefs. Note how, in that case, we trick BibTEXfor inputing extra files twice: we include then with their short name first (with no extension), and then with the full name. Weneedto do that, sincestring abbreviations must be defined first, while crossrefs must occur after having been referenced.

130h∗scripti

131if [ -z "${EXT}" ]; then

132 if [ -z "${EXTRA}" ]; then

133 cat > ${TMPFILE}.aux <<EOF

134\citation{*}

135\bibdata{${FILE}}

136\bibstyle{${BST}}

137EOF

138 else

(8)

139 cat > ${TMPFILE}.aux <<EOF

140\citation{*}

141\bibdata{${FILE}}

142\bibstyle{expkeys}

143EOF

144 bibtex -min-crossrefs=${CREF} -terse ${TMPFILE};

145 mv -f ${TMPFILE}.bbl ${TMPFILE}.aux;

146 cat >> ${TMPFILE}.aux <<EOF

147\bibdata{${EXTRA}${FILE}${EXTRABIB}}

148\bibstyle{${BST}}

149EOF

150 fi

151else

152 cat ${FILE} | sed -e "s/bibstyle{.*}/bibstyle{${BST}}/" > ${TMPFILE}.aux;

153fi

154h/scripti

This was the hard part. We now call BibTEX, clean and rename the output file, and remove intermediary files:

155h∗scripti

156bibtex -min-crossrefs=${CREF} ${TERSE} ${TMPFILE}

157echo "" > ${FINALFILE}

158if [ ! "${NOBANNER}" = "true" ]; then

159 echo -ne "@comment{generated using bibexport:\n" >> ${FINALFILE};

160 echo -ne " creation date:\t‘date +\"%c\"‘\n" >> ${FINALFILE};

161 echo -ne " command:\t\t$0 ${ARGS}\n" >> ${FINALFILE};

162 echo -ne " bibexport-version:\tv${VERSION} (${VDATE})\n" >> ${FINALFILE};

163 echo -ne " bibexport-maintainer:\tmarkey(at)lsv.ens-cachan.fr\n" >> ${FINALFILE};

164 sed -ie "s/}/)/g" ${FINALFILE};

165 echo -ne "}\n\n\n" >> ${FINALFILE};

166fi

167if [ ! "x${CREF}" = "x1" ]; then

168 sed -r -e \

169 "/^ *[cC][rR][oO][sS][sS][rR][eE][fF] *= *[^,]+,?$/d" \

170 ${TMPFILE}.bbl >> ${FINALFILE};

171else

172 cat ${TMPFILE}.bbl >> ${FINALFILE};

173fi

174rm -f ${TMPFILE}.bbl ${TMPFILE}.aux ${TMPFILE}.blg

175h/scripti

2.2 The expkeys.bst file

The only role of that file is to export the list of entries to be exported. It is used when we export all the entries of.bibfiles, except those ofextra .bibfiles. Thus:

176h∗expkeysi

177ENTRY{}{}{}

178READ

179FUNCTION{export.key}

(9)

180{

181 "\citation{" cite$ "}" * * write$ newline$

182}

183ITERATE{export.key}

184h/expkeysi

2.3 The export.bst file

2.3.1 Some configuration values

left.width right.width url.right.width left.short.width right.short.width left.delim right.delim

We define here the indentation values, and the field delimiters. short width are used for@preamble.

185h∗exporti

186FUNCTION{left.width}{#18}

187FUNCTION{right.width}{#55}

188FUNCTION{url.right.width}{#61}

189FUNCTION{left.short.width}{#10} %% for @preamble

190FUNCTION{right.long.width}{#63}

191FUNCTION{left.delim}{"{"}

192FUNCTION{right.delim}{"}"}

193%FUNCTION{left.delim}{quote$}

194%FUNCTION{right.delim}{quote$}

195h/exporti

2.3.2 Entries

We use standard entries here. Of course, more entries could be added for special .bib files. Those extra entries will also have to be added in the main exporting function.

ENTRY

196h∗exporti

197ENTRY{

198% Standard fields:

199 address

200 author

201 booktitle

202 chapter

203 edition

204 editor

205 howpublished

206 institution

207 journal

208 key

209 month

210 note

211 number

212 organization

213 pages

(10)

214 publisher

215 school

216 series

217 title

218 type

219 volume

220 year

221% Special (but still somewhat standard) fields (natbib, germbib, ...):

222 abstract

223 doi

224 eid

225 isbn

226 issn

227 language

228 url

229}{}{}

230h/exporti

2.3.3 Basic functions

No comment.

or and not

231h∗exporti

232FUNCTION{not}

233{

234 {#0}

235 {#1}

236 if$

237}

238FUNCTION{and}

239{

240 ’skip$

241 {pop$ #0}

242 if$

243}

244FUNCTION{or}

245{

246 {pop$ #1}

247 ’skip$

248 if$

249}

250h/exporti

2.3.4 Splitting strings

We design functions for splitting strings, so that the final.bibfile will be cleanly indented.

space.complete split.string split.url

(11)

251h∗exporti

252INTEGERS{left.length right.length}

253STRINGS{ s t }

254FUNCTION{space.complete}

255{

256 ’left.length :=

257 duplicate$ text.length$ left.length swap$ -

258 {duplicate$ #0 >}

259 {

260 swap$ " " * swap$ #1 -

261 }

262 while$

263 pop$

264}

265FUNCTION{split.string}

266{

267 ’right.length :=

268 duplicate$ right.length #1 + #1 substring$ "" =

269 {""}

270 {

271 ’s :=

272 right.length

273 {duplicate$ duplicate$ s swap$ #1 substring$ " " = not and}

274 {#1 -}

275 while$

276 duplicate$ #2 <

277 {

278 pop$ " " s * ""

279 }

280 {

281 duplicate$ s swap$ #1 swap$ substring$

282 swap$

283 s swap$ global.max$ substring$

284 }

285 if$

286 }

287 if$

288}

289FUNCTION{split.url}

290{

293 {""}

294 {

295 ’s :=

296 right.length

297 {duplicate$ duplicate$ s swap$ #1 substring$ "/" = not and}

298 {#1 -}

299 while$

300 duplicate$ #2 <

(12)

301 {

302 pop$ " " s * ""

303 }

304 {

305 duplicate$ s swap$ #1 swap$ substring$

306 swap$ #1 +

308 }

309 if$

310 }

311 if$

312}

313FUNCTION{split.name}

314{

317 {""}

318 {

319 ’s :=

320 right.length

321 {duplicate$ duplicate$ s swap$ #5 substring$ " and " = not and}

322 {#1 -}

323 while$

324 duplicate$ #2 <

325 {

326 pop$ " " s * ""

327 }

328 {

329 #4 + duplicate$ s swap$ #1 swap$ substring$

330 swap$

332 }

333 if$

334 }

335 if$

336}

337h/exporti

2.3.5 Exporting fields

Here, we have four exporting functions, since we also have to deal with abbreviations:

field.export abbrv.export name.export url.export

338h∗exporti

339FUNCTION{field.export}

340{

341 duplicate$ missing$

342 ’skip$

343 {

(13)

344 left.delim swap$ * right.delim *

345 swap$

346 " " swap$ * " = " * left.width space.complete

347 swap$ "," *

348 {duplicate$ "" = not}

349 {

350 right.width split.string ’t :=

351 *

352 write$ newline$

353 "" left.width space.complete t

354 }

355 while$

356 }

357 if$

358 pop$ pop$

359}

360FUNCTION{abbrv.export}

361{

363 ’skip$

364 {

365 swap$

367 swap$ "," *

369 {

370 right.width split.string ’t :=

371 *

372 write$ newline$

374 }

375 while$

376 }

377 if$

378 pop$ pop$

379}

380FUNCTION{name.export}

381{

383 ’skip$

384 {

386 swap$

388 swap$ "," *

390 {

391 right.width split.name ’t :=

392 *

393 write$ newline$

(14)

395 }

396 while$

397 }

398 if$

399 pop$ pop$

400}

401FUNCTION{url.export}

402{

404 ’skip$

405 {

407 swap$

409 swap$ "," *

411 {

412 url.right.width split.url ’t :=

413 *

414 write$ newline$

416 }

417 while$

418 }

419 if$

420 pop$ pop$

421}

422h/exporti

2.3.6 Handling abbreviations

Abbreviations are difficult to deal with if we wish to still use them, since BibTEXwill expand them before we can do anything. All we can do is to define them in a special way, in order to be able to get back to the abbreviations later on. This is precisely what we do:

jan-dec acmcs-tcs remove.exports.from.months remove.export.from.journal

423h∗exporti

424MACRO{jan}{"export-jan"}

425MACRO{feb}{"export-feb"}

426MACRO{mar}{"export-mar"}

427MACRO{apr}{"export-apr"}

428MACRO{may}{"export-may"}

429MACRO{jun}{"export-jun"}

430MACRO{jul}{"export-jul"}

431MACRO{aug}{"export-aug"}

432MACRO{sep}{"export-sep"}

433MACRO{oct}{"export-oct"}

434MACRO{nov}{"export-nov"}

(15)

435MACRO{dec}{"export-dec"}

436MACRO{acmcs}{"export-acmcs"}

437MACRO{acta}{"export-acta"}

438MACRO{cacm}{"export-cacm"}

439MACRO{ibmjrd}{"export-ibmjrd"}

440MACRO{ibmsj}{"export-ibmsj"}

441MACRO{ieeese}{"export-ieeese"}

442MACRO{ieeetc}{"export-ieeetc"}

443MACRO{ieeetcad}{"export-ieeetcad"}

444MACRO{ipl}{"export-ipl"}

445MACRO{jacm}{"export-jacm"}

446MACRO{jcss}{"export-jcss"}

447MACRO{scp}{"export-scp"}

448MACRO{sicomp}{"export-sicomp"}

449MACRO{tocs}{"export-tocs"}

450MACRO{tods}{"export-tods"}

451MACRO{tog}{"export-tog"}

452MACRO{toms}{"export-toms"}

453MACRO{toois}{"export-poois"}

454MACRO{toplas}{"export-toplas"}

455MACRO{tcs}{"export-tcs"}

456INTEGERS{ intxt }

457FUNCTION{remove.exports.from.months}

458{

459 #0 ’intxt :=

461 ’skip$

462 {’t :=

463 ""

464 {t #1 #1 substring$ "" = not}

465 {

466 t #1 #7 substring$ "export-" =

467 {intxt

468 {right.delim * #0 ’intxt :=}

469 ’skip$

470 if$

471 duplicate$ "" =

472 ’skip$

473 {" # " *}

474 if$

475 t #8 #3 substring$ *

476 t #11 global.max$ substring$ ’t :=}

477 {intxt

478 ’skip$

479 {duplicate$ "" =

480 {}

481 {" # " *}

482 if$

483 left.delim * #1 ’intxt :=}

484 if$

(16)

485 t #1 #1 substring$ *

486 t #2 global.max$ substring$ ’t :=}

487 if$

488 }

489 while$

490 intxt

491 {right.delim *}

492 ’skip$

493 if$

494 }

495 if$

496}

497FUNCTION{remove.export.from.journals}

498{

500 ’skip$

501 {

502 duplicate$ #1 #7 substring$ "export-" =

503 {#8 global.max$ substring$}

504 {left.delim swap$

505 right.delim * *}

506 if$

507 }

508 if$

509}

510h/exporti

2.3.7 Now, we export...

We gather everything. This is were special fields must be added for being exported:

entry.export.standard entry.export.extra entry.export export

511h∗exporti

512FUNCTION{entry.export.standard}

513{

514 "address" address field.export

515 "author" author name.export

516 "booktitle" booktitle field.export

517 "chapter" chapter field.export

518 "crossref" crossref field.export

519 "edition" edition field.export

520 "editor" editor name.export

521 "howpublished" howpublished field.export

522 "institution" institution field.export

523 "journal" journal remove.export.from.journals abbrv.export

524 "key" key field.export

525 "month" month remove.exports.from.months abbrv.export

526 "note" note field.export

527 "number" number field.export

528 "organization" organization field.export

(17)

529 "pages" pages field.export

530 "publisher" publisher field.export

531 "school" school field.export

532 "series" series field.export

533 "type" type field.export

534 "title" title field.export

535 "volume" volume field.export

536 "year" year field.export

537}

538FUNCTION{entry.export.extra}

539{

540 "abstract" abstract field.export

541 "doi" doi field.export

542 "eid" eid field.export

543 "isbn" isbn field.export

544 "issn" issn field.export

545 "language" language field.export

546 "url" url url.export

547}

548FUNCTION{entry.export}

549{

550 entry.export.standard

551 entry.export.extra

552}

553FUNCTION{export}

554{

555 "@" type$ * "{" * cite$ * "," * write$ newline$

556 entry.export

557 "}" write$ newline$ newline$

558}

559h/exporti

2.3.8 Miscellanea

We also have to handle preamble, and to define functions for each entry type (we won’t use them but otherwise, BibTEXwould complain).

preamble header entries.headers article-unpublished

560h∗exporti

561FUNCTION{preamble}

562{

563preamble$ duplicate$ "" =

564 ’pop$

565 {

566 ",---." write$ newline$

567 "| PREAMBLE |" write$ newline$

568 "‘---’" write$ newline$ newline$

569 "@preamble{ " swap$

570 quote$ swap$ * quote$ *

(18)

572 {

573 right.long.width split.string ’t :=

574 *

575 write$ newline$

576 "" left.short.width space.complete t

577 }

578 while$

579 "}" write$ newline$ newline$

580 pop$ pop$

581 }

582if$

583}

584FUNCTION{header}

585{

586%"** This file has been automatically generated by bibexport **"

587%write$ newline$

588%"** See http://www.lsv.ens-cachan.fr/~markey/bibla.php **"

589%write$ newline$

590%"** for more informations about bibexport. **"

591%write$ newline$

592newline$

593}

594FUNCTION{entries.header}

595{

596preamble$ "" =

597 ’skip$

598 {

599 ",---." write$ newline$

600 "| BIBTEX ENTRIES |" write$ newline$

601 "‘---’" write$ newline$ newline$

602 }

603if$

604}

605FUNCTION{article}{export}

606FUNCTION{book}{export}

607FUNCTION{booklet}{export}

608FUNCTION{conference}{export}

609FUNCTION{habthesis}{export}

610FUNCTION{inbook}{export}

611FUNCTION{incollection}{export}

612FUNCTION{inproceedings}{export}

613FUNCTION{journals}{export}

614FUNCTION{manual}{export}

615FUNCTION{mastersthesis}{export}

616FUNCTION{misc}{export}

617FUNCTION{phdthesis}{export}

618FUNCTION{proceedings}{export}

619FUNCTION{techreport}{export}

620FUNCTION{unpublished}{export}

621h/exporti

(19)

2.3.9 Main program

We now can execute and iterate those functions:

622h∗exporti

623READ

624EXECUTE{header}

625EXECUTE{preamble}

626EXECUTE{entries.header}

627ITERATE{export}

628h/exporti