Catalogue Search | MBRL
Are you sure you want to remove the book from the shelf?
{{itemTitle}}
41,402
result(s) for
"Data reduction"
Sort by:
APERO: A PipelinE to Reduce Observations—Demonstration with SPIRou
With the maturation of near-infrared high-resolution spectroscopy, especially when used for precision radial velocity, data reduction has faced unprecedented challenges in terms of how one goes from raw data to calibrated, extracted, and corrected data with required precisions of thousandths of a pixel. Here we present A PipelinE to Reduce Observations ( apero ), specifically focused on Spectro Polarimètre Infra ROUge (SPIR ou ), the near-infrared spectropolarimeter on the Canada–France–Hawaii Telescope (SPectropolarimètre InfraROUge, CFHT). In this paper, we give an overview of apero and detail the reduction procedure for SPIR ou . apero delivers telluric-corrected 2D and 1D spectra as well as polarimetry products. apero enables precise stable radial velocity measurements on the sky (via the LBL algorithm), which is good to at least ∼2 m s −1 over the current 5 yr lifetime of SPIR ou .
Journal Article
Robust Methods for Data Reduction
This book gives a non-technical overview of robust data reduction techniques, encouraging the use of these important and useful methods in practical applications. The main areas covered include principal components analysis, sparse principal component analysis, canonical correlation analysis, factor analysis, clustering, double clustering, and discriminant analysis. Using real examples, the authors show how to implement the procedures in R. The code and data for the examples are available on the book's CRC Press web page.
eBook
SImMER: A Pipeline for Reducing and Analyzing Images of Stars
We present the first public version of SImMER , an open-source Python reduction pipeline for astronomical images of point sources. Current capabilities include dark-subtraction, flat-fielding, sky-subtraction, image registration, FWHM measurement, contrast curve calculation, and table and plot generation. SImMER supports observations taken with the ShARCS camera on the Shane 3 m telescope and the PHARO camera on the Hale 5.1 m telescope. The modular nature of SImMER allows users to extend the pipeline to accommodate additional instruments with relative ease. One of the core functions of the pipeline is its image registration module, which is flexible enough to reduce saturated images and images of similar-brightness, resolved stellar binaries. Furthermore, SImMER can compute contrast curves for reduced images and produce publication-ready plots. The code is developed online at https://github.com/arjunsavel/SImMER and is both pip- and conda-installable. We develop tutorials and documentation alongside the code and host them online. With SImMER , we aim to provide a community resource for accurate and reliable data reduction and analysis.
Journal Article
The Near Infrared Imager and Slitless Spectrograph for JWST. V. Kernel Phase Imaging and Data Analysis
Kernel phase imaging (KPI) enables the direct detection of substellar companions and circumstellar dust close to and below the classical (Rayleigh) diffraction limit. The high-Strehl full pupil images provided by the James Webb Space Telescope (JWST) are ideal for application of the KPI technique. We present a kernel phase analysis of JWST NIRISS full pupil images taken during the instrument commissioning and compare the performance to closely related NIRISS aperture masking interferometry (AMI) observations. For this purpose, we develop and make publicly available the custom Kpi3Pipeline data reduction pipeline enabling the extraction of kernel phase observables from JWST images. The extracted observables are saved into a new and versatile kernel phase FITS file data exchange format. Furthermore, we present our new and publicly available fouriever toolkit which can be used to search for companions and derive detection limits from KPI, AMI, and long-baseline interferometry observations while accounting for correlated uncertainties in the model fitting process. Among the four KPI targets that were observed during NIRISS instrument commissioning, we discover a low-contrast (∼1:5) close-in (∼1 λ / D ) companion candidate around CPD-66 562 and a new high-contrast (∼1:170) detection separated by ∼1.5 λ / D from 2MASS J062802.01-663738.0. The 5 σ companion detection limits around the other two targets reach ∼6.5 mag at ∼200 mas and ∼7 mag at ∼400 mas. Comparing these limits to those obtained from the NIRISS AMI commissioning observations, we find that KPI and AMI perform similar in the same amount of observing time. Due to its 5.6 times higher throughput if compared to AMI, KPI is beneficial for observing faint targets and superior to AMI at separations ≳325 mas. At very small separations (≲100 mas) and between ∼250 and 325 mas, AMI slightly outperforms KPI which suffers from increased photon noise from the core and the first Airy ring of the point-spread function.
Journal Article
Asteroid Photometry with PIRATE: Optimizations and Techniques for Small Aperture Telescopes
Small aperture telescopes provide the opportunity to conduct high frequency, targeted observations of near-Earth Asteroids that are not feasible with larger facilities due to highly competitive time allocation requirements. Observations of asteroids with these types of facilities often focus on rotational brightness variations rather than longer-term phase angle-dependent variations (phase curves) due to the difficulty of achieving high precision photometric calibration. We have developed an automated asteroid light curve extraction and calibration pipeline for images of moving objects from the 0.43 m Physics Innovations Robotic Telescope Explorer. This allows for the frequency and quality of observations required to construct asteroid phase curves. Optimizations in standard data reduction procedures are identified that may allow for similar small aperture facilities, constructed from commercially available/off-the-shelf components, to improve the image and subsequent data quality. A demonstration of the hardware and software capabilities is expressed through observation statistics from a 10 months observing campaign, and through the photometric characterization of near-Earth Asteroids 8014 (1990 MF) and 19764 (2000 NF5).
Journal Article
Dimensionality Reduction and Classification of Hyperspectral Remote Sensing Image Feature Extraction
Terrain classification is an important research direction in the field of remote sensing. Hyperspectral remote sensing image data contain a large amount of rich ground object information. However, such data have the characteristics of high spatial dimensions of features, strong data correlation, high data redundancy, and long operation time, which lead to difficulty in image data classification. A data dimensionality reduction algorithm can transform the data into low-dimensional data with strong features and then classify the dimensionally reduced data. However, most classification methods cannot effectively extract dimensionality-reduced data features. In this paper, different dimensionality reduction and machine learning supervised classification algorithms are explored to determine a suitable combination method of dimensionality reduction and classification for hyperspectral images. Soft and hard classification methods are adopted to achieve the classification of pixels according to diversity. The results show that the data after dimensionality reduction retain the data features with high overall feature correlation, and the data dimension is drastically reduced. The dimensionality reduction method of unified manifold approximation and projection and the classification method of support vector machine achieve the best terrain classification with 99.57% classification accuracy. High-precision fitting of neural networks for soft classification of hyperspectral images with a model fitting correlation coefficient (R2) of up to 0.979 solves the problem of mixed pixel decomposition.
Journal Article
Improving IoT Security: The Impact of Dimensionality and Size Reduction on Intrusion Detection Performance
Intrusion detection in the Internet of Things (IoT) environments is essential to guarantee computer network security. Machine learning (ML) models are widely used to improve efficient detection systems. Meanwhile, with the increasing complexity and size of intrusion detection data, analyzing vast datasets using ML models is becoming more challenging and demanding in terms of computational resources. Datasets related to IoT environments usually come in very large sizes. This study investigates the impact of dataset reduction techniques on machine learning-based Intrusion Detection Systems (IDS) performance and efficiency. We propose a two-stage framework incorporating deep autoencoder-based feature reduction with stratified sampling to reduce the dimensionality and size of six publicly available IDS datasets, including BoT-IoT, CSE-CIC-IDS2018, and others. Multiple machine learning models, such as Random Forest, XGBoost, K-Nearest Neighbors, SVM, and AdaBoost, were evaluated using default parameters. Our results show that dataset reduction can decrease training time by up to 99% with minimal loss in F1-score, typically less than 1%. It is recognized that excessive size reduction can compromise detection accuracy for minority attack classes. However, employing a stratified sampling method can effectively maintain class distributions. The study highlights significant feature redundancy, particularly high correlation among features, across multiple IoT security-related datasets, motivating the use of dimensionality reduction techniques. These findings support the feasibility of efficient, scalable IDS implementations for real-world environments, especially in resource-constrained or real-time settings. [JJCIT 2025; 11(3.000): 351-368]
Journal Article
KMTNet Nearby Galaxy Survey: Overview and Survey Description
Recently, there has been increasing demand for deep imaging surveys to investigate the history of the mass assembly of galaxies in detail by examining the remnants of mergers and accretions, both of which have very low surface brightness (LSB). In addition, the nature of star formation in LSB regions, such as galaxy outer disks, is also an intriguing topic in terms of understanding the physical mechanisms of disk evolution. To address these issues, this study conducted a survey project, called the Korea Microlensing Telescope Network Nearby Galaxy Survey, to construct a deep imaging data set of nearby galaxies in the southern hemisphere. It provides deep and wide-field images with a field of view of ∼12 deg 2 for 13 nearby galaxies drawn from the Carnegie–Irvine Galaxy Survey catalog in optical broad bands ( BRI ) and an H α narrow band. Through a dedicated data reduction, the surface brightness limit in 10″ × 10″ boxes was found to reach as deep as μ 1 σ ∼ 29–31 mag arcsec −2 in the optical broad bands and f 1 σ ∼ 1–2 × 10 −18 erg s −1 cm −2 arcsec −2 in the H α narrow band. To conclude the paper, several possible scientific applications for this data set are described.
Journal Article
Innovative data reduction and visualization strategy for hyperspectral imaging datasets using t-SNE approach
Visible hyperspectral imaging (HSI) is a fast and non-invasive imaging method that has been adapted by the field of conservation science to study painted surfaces. By collecting reflectance spectra from a 2D surface, the resulting 3D hyperspectral data cube contains millions of recorded spectra. While processing such large amounts of spectra poses an analytical and computational challenge, it also opens new opportunities to apply powerful methods of multivariate analysis for data evaluation. With the intent of expanding current data treatment of hyperspectral datasets, an innovative approach for data reduction and visualization is presented in this article. It uses a statistical embedding method known as t-distributed stochastic neighbor embedding (t-SNE) to provide a non-linear representation of spectral features in a lower 2D space. The efficiency of the proposed method for painted surfaces from cultural heritage is established through the study of laboratory prepared paint mock-ups, and medieval French illuminated manuscript.
Journal Article