LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop

Fisher YuAri SeffYinda ZhangShuran SongThomas FunkhouserJianxiong Xiao

article2015arXiv2,698 citations

Presents an efficient human-in-the-loop labeling framework that scales dataset construction using deep model predictions, producing the massive LSUN visual recognition benchmark spanning millions of scene and object images.

Listen

The article addresses the challenge of creating sufficiently large and dense labeled image datasets to train high-capacity deep learning models for visual recognition. Existing collections such as ImageNet and Places have become too small relative to modern network sizes, and fully manual labeling cannot scale fast enough to close the gap.

The work set out to build a new dataset, LSUN, containing roughly one million labeled images in each of ten scene categories and twenty object categories, while keeping human effort practical.

Researchers collected more than one hundred million candidate images per category through web search, then applied an iterative pipeline. In each round a modest number of images received manual labels from Amazon Mechanical Turk workers; those labels trained a convolutional classifier that confidently labeled easy positives and negatives and passed only ambiguous images to the next round. The process continued until the remaining set was small enough for exhaustive manual review. Quality was maintained through redundant labeling, tutorial checks, and hidden ground-truth images.

The resulting dataset contains ten million scene images and fifty-nine million object images. Average labeling precision reached approximately ninety percent. Human effort was amplified by a factor of roughly forty. When AlexNet was fine-tuned on the new data, classification error on a Places-derived test set fell from 28.6 percent to 22.2 percent. Pre-training the same architecture on LSUN object images raised mean average precision on PASCAL VOC 2012 from 0.80 to 0.87; a deeper VGG network showed a similar gain.

These outcomes indicate that training on larger but slightly noisy data can outperform training on smaller clean collections, and that first-layer filters learned from the denser set appear cleaner and more coherent. The gains matter because they demonstrate a practical route to continued progress in visual recognition without proportional increases in manual annotation cost.

The authors plan to keep expanding LSUN and to release it publicly. Practitioners should consider using the data for pre-training or fine-tuning when category coverage aligns with their needs, while monitoring for residual label noise on categories that contain many toys or rendered images. Additional validation on new tasks and architectures would strengthen before large-scale deployment.

The main limitations are the ninety-percent precision ceiling, possible dataset bias relative to other collections, and the focus on only two model families. Results should be treated as indicative rather than definitive for every downstream application.

arXiv: 1506.03365fyu/lsun
  • Paper: Learning Deep Features for Scene Recognition using Places Database, Bolei Zhou et al. (2014). It introduces the Places database and demonstrates how dedicated large-scale scene datasets enable convolutional networks to learn scene-centric features, directly motivating LSUN's creation and evaluation protocol.
  • Paper: ImageNet: A large-scale hierarchical image database, Jia Deng et al. (2009). It establishes the foundational methodology of using Amazon Mechanical Turk crowdsourcing to construct large-scale image ontologies that LSUN scales up through active human-in-the-loop filtering.
  • Paper: ImageNet Classification with Deep Convolutional Neural Networks, Alex Krizhevsky et al. (2012). It provides the baseline AlexNet architecture and pretraining methodology that LSUN fine-tunes to demonstrate the empirical benefits of large-scale scene data.
  • Paper: Unbiased look at dataset bias, A. Torralba et al. (2011). It formalizes the problem of dataset bias across visual recognition benchmarks, a key analytical limitation and motivation evaluated in the LSUN paper.
  • Paper: Microsoft COCO: Common Objects in Context, Tsung-Yi Lin et al. (2014). It provides the baseline pipeline for multi-stage crowdsourced visual data collection and standard object detection benchmarks used for downstream validation in LSUN.
Cover for LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop

Abstract

While there has been remarkable progress in the performance of visual recognition algorithms, the state-of-the-art models tend to be exceptionally data-hungry. Large labeled training datasets, expensive and tedious to produce, are required to optimize millions of parameters in deep network models. Lagging behind the growth in model capacity, the available datasets are quickly becoming outdated in terms of size and density. To circumvent this bottleneck, we propose to amplify human effort through a partially automated labeling scheme, leveraging deep learning with humans in the loop. Starting from a large set of candidate images for each category, we iteratively sample a subset, ask people to label them, classify the others with a trained model, split the set into positives, negatives, and unlabeled based on the classification confidence, and then iterate with the unlabeled set. To assess the effectiveness of this cascading procedure and enable further progress in visual recognition research, we construct a new image dataset, LSUN. It contains around one million labeled images for each of 10 scene categories and 20 object categories. We experiment with training popular convolutional networks and find that they achieve substantial performance gains when trained on this dataset.

Table of Contents

  • 1 Introduction
  • 2 Overview
  • 3 Data Collection
  • 4 Deep Learning with Humans in the Loop
  • 5 Crowd Sourcing
  • 6 Results
  • 7 Conclusion
  • References

Knowls

  1. Knowl 1 — Cascaded Semi-Automated Dataset Annotation Pipeline

    algorithm

    The human-in-the-loop annotation pipeline expands candidate image sets into accurately labeled categories by iteratively learning a cascade of targeted, over-fit binary classifiers. In each round, human crowd annotators label a small subset of the remaining unlabeled pool. The classifier trained on accumulated human labels partitions the unlabeled dataset into confident positives (added to the target dataset), confident negatives (permanently discarded), and ambiguous images. Ambiguous images are passed to the next iteration along with newly sampled images, progressively concentrating human effort on difficult decision boundaries until the residual set is small enough for exhaustive manual annotation.

    Input: Pool of unlabeled candidate images UU, target positive precision ppos=0.95p_{\text{pos}} = 0.95, target negative omission rate rneg=0.01r_{\text{neg}} = 0.01, sample batch size N=40000N = 40000
    Output: Labeled positive set PP, discarded negative set NdiscardN_{\text{discard}}
    PP \leftarrow \emptyset
    NdiscardN_{\text{discard}} \leftarrow \emptyset
    DtrainD_{\text{train}} \leftarrow \emptyset
    while UU is not small enough for exhaustive manual labeling do
        Sample NN images SUS \subset U uniformly at random
        Partition SS into StestS_{\text{test}} (10,000 images), SvalS_{\text{val}} (5,000 images), and SsubtrainS_{\text{subtrain}} (25,000 images)
        Obtain doubly confirmed crowd labels for StestS_{\text{test}}, SvalS_{\text{val}}, and SsubtrainS_{\text{subtrain}} via AMT
        DtrainDtrainSsubtrainD_{\text{train}} \leftarrow D_{\text{train}} \cup S_{\text{subtrain}}
        Train classifier fθf_\theta on DtrainD_{\text{train}} and select best checkpoint on SvalS_{\text{val}}
        Evaluate fθ(x)f_\theta(x) for all xStestx \in S_{\text{test}}
        Compute score threshold τpos\tau_{\text{pos}} such that 95%95\% of samples in StestS_{\text{test}} with fθ(x)τposf_\theta(x) \ge \tau_{\text{pos}} are true positives
        Compute score threshold τneg\tau_{\text{neg}} such that only 1%1\% of all true positives in StestS_{\text{test}} have fθ(x)<τnegf_\theta(x) < \tau_{\text{neg}}
        for each xUx \in U do
            if fθ(x)τposf_\theta(x) \ge \tau_{\text{pos}} then
                PP{x}P \leftarrow P \cup \{x\}
                UU{x}U \leftarrow U \setminus \{x\}
            else if fθ(x)<τnegf_\theta(x) < \tau_{\text{neg}} then
                NdiscardNdiscard{x}N_{\text{discard}} \leftarrow N_{\text{discard}} \cup \{x\}
                UU{x}U \leftarrow U \setminus \{x\}
            end if
        end for
        Retain ambiguous labeled samples from SS in DtrainD_{\text{train}} for subsequent iterations
    end while
    Manually label all remaining images in UU and assign to PP or NdiscardN_{\text{discard}}
    return P,NdiscardP, N_{\text{discard}}
  2. Knowl 2 — Dual-Threshold Empirical Confidence Partitioning

    model/method

    To automatically partition candidate images while guaranteeing dataset quality, the classification score distribution of a trained model f(x)f(x) is evaluated on an independently labeled validation test split StestS_{\text{test}}. Two statistical thresholds, τpos\tau_{\text{pos}} and τneg\tau_{\text{neg}}, are empirically determined:

    1. Positive Threshold (τpos\tau_{\text{pos}}): Defined as the classifier score value such that among all instances in StestS_{\text{test}} scoring at or above τpos\tau_{\text{pos}}, 95%95\% are ground-truth positive annotations: xStestI(f(x)τposyx=1)xStestI(f(x)τpos)=0.95\frac{\sum_{x \in S_{\text{test}}} \mathbb{I}(f(x) \ge \tau_{\text{pos}} \land y_x = 1)}{\sum_{x \in S_{\text{test}}} \mathbb{I}(f(x) \ge \tau_{\text{pos}})} = 0.95 where yx{0,1}y_x \in \{0, 1\} is the human ground-truth label for image xx.

    2. Negative Threshold (τneg\tau_{\text{neg}}): Defined as the classifier score value such that no more than 1%1\% of the total true positive images in StestS_{\text{test}} score strictly below τneg\tau_{\text{neg}}: xStestI(f(x)<τnegyx=1)xStestI(yx=1)=0.01\frac{\sum_{x \in S_{\text{test}}} \mathbb{I}(f(x) < \tau_{\text{neg}} \land y_x = 1)}{\sum_{x \in S_{\text{test}}} \mathbb{I}(y_x = 1)} = 0.01

    Any image in the unlabeled pool scoring f(x)τposf(x) \ge \tau_{\text{pos}} is automatically assigned a positive label and included in the dataset. Any image scoring f(x)<τnegf(x) < \tau_{\text{neg}} is classified as negative and discarded. Images with scores τnegf(x)<τpos\tau_{\text{neg}} \le f(x) < \tau_{\text{pos}} are retained as the ambiguous subset and forwarded to subsequent labeling iterations.

  3. Knowl 3 — Crowdsourced Verification Interface and Anti-Cheating Quality Control

    model/method

    Acquiring annotations from Amazon Mechanical Turk (AMT) relies on structured Human Intelligence Tasks (HITs) and multi-layered verification checks:

    • HIT Structure: Each HIT comprises 205 fullscreen images displayed sequentially with category definitions, previous/next thumbnails, and binary toggle keybindings. Of these, 150 are candidate images and 55 are embedded control images.
    • Redundant Doubly-Confirmed Labeling: Every image is independently labeled by two separate workers; only images with unanimous agreement are retained as ground truth.
    • Tutorial and Gatekeeping (15 images): The initial 15 images of each HIT test comprehension of edge cases. Incorrect labels trigger modal pop-ups blocking progression until rectified.
    • Online Ground Truth Check (20 images): Twenty embedded images with pre-verified labels are evaluated client-side. Workers failing to reach 90%\ge 90\% accuracy cannot submit the HIT and are prompted to review their labels.
    • Hidden Ground Truth Check (20 images): Twenty ground truth images are validated server-side after submission without exposing their labels to the browser DOM, requiring 85%\ge 85\% accuracy to prevent script manipulation or selective online-check bypass.
  4. Knowl 4 — Large-Scale Scene Understanding (LSUN) Dataset Structure

    definition

    The Large-Scale Scene Understanding (LSUN) dataset is a high-density visual benchmark designed to scale up the number of instances per visual concept. It contains labeled natural images across two domains:

    • Scene Categories: 10 scene classes selected from the SUN database (bedroom, kitchen, living room, dining room, bridge, tower, restaurant, conference room, classroom, church outdoor), containing approximately 1 million labeled images per category (totaling 10 million scene images).
    • Object Categories: 20 object classes matching PASCAL VOC 2012 (aeroplane, bicycle, bird, boat, bottle, bus, car, cat, chair, cow, dining table, dog, horse, motorbike, person, potted plant, sheep, sofa, train, tv/monitor), totaling 59 million labeled object images.

    LSUN is approximately 10 times denser per category than the Places database and 100 times denser per basic category than ImageNet.

  5. Knowl 5 — Search Query Generation via Adjective Augmentation and Time Slicing

    model/method

    To harvest over 100 million candidate image URLs per category while circumventing commercial search result quotas, web image queries (via Google Images) are expanded using two complementary techniques:

    1. Adjective Augmentation: Category names are systematically combined with extensive lists of descriptive English adjectives (696 scene-relevant adjectives such as messy, sunny, desolate for scene categories, and dedicated object-relevant adjectives for object categories) to diversify visual appearances and broaden candidate retrieval.
    2. Time Slicing: Each adjective-category query combination is restricted to 3-day publication intervals and queried exhaustively for all 3-day time spans from 2009 onward.

    Harvested images undergo minimum-dimension filtering, retaining only images where min(height,width)>256\min(\text{height}, \text{width}) > 256 pixels (retaining roughly 60%60\% of downloads). Near-duplicate images with varied compression artifacts, crops, and resolutions are retained in the initial pool to serve as natural visual augmentations.

  6. Knowl 6 — Transfer Classification Performance: LSUN vs. ImageNet Pre-Training on PASCAL VOC 2012

    data/table

    Pre-training deep convolutional networks on high-density category data (LSUN object categories sampled at 300,000 images per class across 20 classes) produces representations that generalize better to downstream classification tasks than pre-training on broader datasets with fewer examples per category (ImageNet with 1,000 classes). Models fine-tuned on the PASCAL VOC 2012 classification training set with Hinge loss and evaluated using 10-crop averaging on the PASCAL VOC 2012 validation set achieve consistently higher mean Average Precision (mAP) when pre-trained on LSUN.

    Model Pre-train aero bike bird boat bottle bus car cat chair cow table dog horse mbike person plant sheep sofa train tv mAP
    AlexNet ImageNet 0.96 0.86 0.88 0.85 0.49 0.94 0.75 0.89 0.62 0.86 0.65 0.87 0.92 0.90 0.85 0.53 0.91 0.68 0.92 0.77 0.80
    AlexNet LSUN 0.98 0.93 0.94 0.90 0.64 0.95 0.78 0.97 0.74 0.96 0.72 0.96 0.98 0.94 0.85 0.59 0.96 0.76 0.97 0.82 0.87
    VGG ImageNet 0.95 0.85 0.92 0.86 0.65 0.92 0.80 0.93 0.68 0.77 0.66 0.90 0.85 0.86 0.94 0.56 0.89 0.60 0.94 0.85 0.82
    VGG LSUN 0.98 0.92 0.94 0.90 0.69 0.96 0.82 0.96 0.75 0.97 0.78 0.93 0.97 0.92 0.95 0.67 0.96 0.73 0.97 0.84 0.88

    AlexNet pre-trained on LSUN demonstrates an overall mAP gain of +0.07+0.07 (0.87 vs. 0.80) compared to ImageNet pre-training. Similarly, VGG achieves +0.06+0.06 mAP improvement (0.88 vs. 0.82).

  7. Knowl 7 — Scene Classification Accuracy Improvement on PLACES from Adding LSUN Data

    empirical result

    Augmenting scene classification training data with LSUN images significantly reduces classification error on the standard PLACES benchmark. Evaluating an AlexNet architecture on the 10 shared scene categories of the PLACES test set yields the following results:

    • Fine-tuning AlexNet on PLACES alone results in an overall test classification error rate of 28.6%28.6\%.
    • Fine-tuning AlexNet on PLACES combined with LSUN (capped at 200,000 images per category from LSUN) reduces the overall test error rate to 22.2%22.2\%.
    • This corresponds to a 22.37%22.37\% relative reduction in testing error, with classification accuracy improving across 8 of the 10 individual scene categories (bedroom, kitchen, living room, dining room, bridge, tower, restaurant, and church outdoor).
  8. Knowl 8 — Precision and Human Effort Amplification Rates in Semi-Automated Annotation

    empirical result

    Validation experiments evaluating the quality and efficiency of the semi-automated human-in-the-loop pipeline demonstrate:

    • Label Precision: On an evaluation set of 2,000 images sampled across 11 object categories and labeled by trained expert annotators, average labeling precision is approximately 90%90\%. The primary sources of label noise are toy representations of objects, computer-rendered 3D models (especially within the person category), photo collages, and heavily edited imagery.
    • Effort Amplification: The ratio of total images accurately categorized (assigned positive or negative) to the number of images explicitly labeled by human annotators averages 40:140:1 across categories, exceeding 60:160:1 for categories with distinctive visual features such as train.
  9. Knowl 9 — First-Layer Convolutional Filter Quality under High Data Density

    empirical result

    Visual inspection of first-layer convolutional filters in AlexNet models reveals distinct qualitative differences depending on dataset density:

    • Filters learned from scratch on ImageNet (broad coverage of 1,000 categories with ~1,000 images per class) exhibit substantial high-frequency noise, checkerboard artifacts, and less coherent edge orientations.
    • Filters learned from scratch on LSUN object images (dense coverage of 20 categories with hundreds of thousands of images per class) converge to noticeably cleaner, smoother Gabor-like edge detectors and well-defined color opponent blobs.
  10. Knowl 10 — Staged Classifier Architecture and Training Schedule in Label Propagation

    experimental setup

    The classification model used in the labeling cascade adapts across iterations to balance computational throughput against model capacity:

    • Iterations 1–2: Features extracted from the final fully connected layer (fc7) of a pre-trained AlexNet are classified using a Multi-Layer Perceptron (MLP) with two hidden layers. This fast classifier eliminates over 50%50\% of easy negatives from the initial candidate pool.
    • Iterations 3+: The pipeline transitions to fine-tuning a pre-trained GoogLeNet architecture using mini-batches of size 80. Fine-tuning uses an initial learning rate of 0.0010.001, decaying to 0.00010.0001 after 40,000 iterations for a total of 60,000 training iterations.
    • Inference Cropping Schedule: In early labeling iterations, inference is computed on single central crops to maximize speed across tens of millions of candidates. After iteration 10, when the unlabeled pool has been substantially reduced, inference evaluates 10 crops per image (center and four corners with horizontal mirroring) and averages the predictions.

Coverage note — None omitted; all primary contributions, including dataset statistics, querying strategies, cascaded labeling algorithms, crowdsourcing mechanisms, downstream transfer results, and filter representations, are fully documented.

References

  1. 1.http://www.image-net.org/challenges/LSVRC/announcement-June-2-2015.
  2. 2.S. Branson, C. Wah, F. Schroff, B. Babenko, P. Welinder, P. Perona, and S. Belongie. Visual recognition with humans in the loop. In ECCV, 2010.
  3. 3.B. Collins, J. Deng, K. Li, and L. Fei-Fei. Towards scalable dataset construction: An active learning approach. In ECCV, pages 86–98. Springer, 2008.
  4. 4.J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248–255. IEEE, 2009.
  5. 5.J. Deng, O. Russakovsky, J. Krause, M. Bernstein, A. Berg, and L. Fei-Fei. Scalable multi-label annotation. In CHI, 2014.
  6. 6.L. Fei-Fei, R. Fergus, and P. Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. CVIU, 106(1):59–70, 2007.
  7. 7.K. He, X. Zhang, S. Ren, and J. Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. arXiv preprint arXiv:1502.01852, 2015.
  8. 8.S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015.
  9. 9.A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012.
  10. 10.Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998.
  11. 11.A. Nguyen, J. Yosinski, and J. Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. arXiv preprint arXiv:1412.1897, 2014.
  12. 12.O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):211–252, 2015.
  13. 13.O. Russakovsky, L.-J. Li, and L. Fei-Fei. Best of both worlds: human-machine collaboration for object annotation. In CVPR, pages 2121–2131, 2015.
  14. 14.B. Settles. Active learning literature survey. University of Wisconsin, Madison, 52(55-66):11, 2010.
  15. 15.K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
  16. 16.C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. arXiv preprint arXiv:1409.4842, 2014.
  17. 17.C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
  18. 18.S. Tong and D. Koller. Support vector machine active learning with applications to text classification. The Journal of Machine Learning Research, 2:45–66, 2002.
  19. 19.A. Torralba and A. A. Efros. Unbiased look at dataset bias. In CVPR, 2011.
  20. 20.S. Vijayanarasimhan and K. Grauman. Multi-level active prediction of useful image annotations for recognition. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, Advances in Neural Information Processing Systems 21, pages 1705–1712. Curran Associates, Inc., 2009.
  21. 21.C. Wah, S. Branson, P. Perona, and S. Belongie. Multiclass recognition and part localization with humans in the loop. In ICCV, 2011.
  22. 22.R. Wu, S. Yan, Y. Shan, Q. Dang, and G. Sun. Deep image: Scaling up image recognition. arXiv preprint arXiv:1501.02876, 2015.
  23. 23.J. Xiao, J. Hays, K. A. Ehinger, A. Oliva, and A. Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In CVPR, pages 3485–3492. IEEE, 2010.
  24. 24.B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva. Learning deep features for scene recognition using places database. In NIPS, 2014.

Citation

MLA
Yu, F., et al. “LSUN: Construction of a Large-scale Image Dataset Using Deep Learning with Humans in the Loop”. arXiv, 2015, https://doi.org/10.48550/arxiv.1506.03365.
APA
Yu, F., Seff, A., Zhang, Y., Song, S., Funkhouser, T., & Xiao, J. (2015). LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop. arXiv. https://doi.org/10.48550/arxiv.1506.03365
Chicago
Yu, F., A. Seff, Y. Zhang, S. Song, T. Funkhouser, and J. Xiao. 2015. “LSUN: Construction of a Large-scale Image Dataset Using Deep Learning with Humans in the Loop”. Preprint, ArXiv. https://doi.org/10.48550/arxiv.1506.03365.
Harvard
Yu, F. et al. (2015) “LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop”. arXiv. Available at: https://doi.org/10.48550/arxiv.1506.03365.
Vancouver
1. Yu F, Seff A, Zhang Y, Song S, Funkhouser T, Xiao J (2015) LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop. https://doi.org/10.48550/arxiv.1506.03365

BibTeX

@misc{https://doi.org/10.48550/arxiv.1506.03365,
  doi = {10.48550/ARXIV.1506.03365},
  url = {https://arxiv.org/abs/1506.03365},
  author = {Yu, Fisher and Seff, Ari and Zhang, Yinda and Song, Shuran and Funkhouser, Thomas and Xiao, Jianxiong},
  keywords = {Computer Vision and Pattern Recognition (cs.CV), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop},
  publisher = {arXiv},
  year = {2015},
  copyright = {arXiv.org perpetual, non-exclusive license}
}
Metadata:DOI registry

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Published with permission