26,663 questions
0
votes
0
answers
71
views
Access non-Iceberg Glue tables cross-account from Spark (Glue job) using AssumeRole
I have a Glue Catalog in Account B containing multiple tables: some are Iceberg, some are standard (non-Iceberg / Hive / external)
I run a Glue Spark job in Account A with RoleA, and I need to access ...
- 1,023
Advice
0
votes
2
replies
44
views
How can I have hive metastore logging to its own file?
I can see cloudera has separate log files for hiveserver and metastore. I am unsure how to make this happen for standalone hive install (no cloudera or ambari). I have tried looking around the ...
- 1
0
votes
1
answer
149
views
Order by column first alpha char then numeric char
When I have order by clause in Hive query, for example:
SELECT *
FROM EPAY
ORDER BY EUNQ
Current Results
14579536
50118664
5PM35795
Expected Results
14579536
5PM35795
50118664
Is there anything I can ...
1
vote
2
answers
91
views
Flutter Hive error: The box "cartbox" is already open and of type Box<Product>
I open the Hive box in main.dart
void main() async {
WidgetsFlutterBinding.ensureInitialized();
await Hive.initFlutter();
Hive.registerAdapter(ProductAdapter());
await Hive.openBox<...
Advice
0
votes
1
replies
67
views
How to configure log4j2 for multiple hive processes
Looking through documentation and articles is not helping me to decipher configuration.
The default hive setup shows
name = HiveLog4j2
# list of properties
property.hive.log.level = INFO
property....
- 1
0
votes
0
answers
71
views
implementing a foreground service with hive -> sequentially sending data to the server
i am implementing a foreground service so that when there is no internet , the data that i directly send to my server should be reside in hive sequentially and when internet comes, the foreground ...
0
votes
0
answers
60
views
msck repair table sync partitions fails
I have a pyspark job that write dataframe to s3 with partitions. the partition value is string. in my pyspark script, I have the line:
spark.sql("MSCK REPAIR TABLE table_name SYNC PARTITIONS"...
- 169
0
votes
1
answer
96
views
Hive Metastore Standalone Java version incompatibility
I have recently donwloaded a standalone hive metastore binary distrubution. I am also running an Apache Hadoop instance on the same machine, and Hive seems to recognize Hadoop. However, when trying to ...
- 557
0
votes
1
answer
170
views
Trino Select from public aws S3 storage anonymously (hive connector)
My hive connector config, hive_aws.properties:
connector.name=hive
hive.metastore.uri=thrift://hive-metastore:9083
fs.native-s3.enabled=true
s3.endpoint=https://s3.amazonaws.com
s3.region=us-east-1
(...
- 353
1
vote
0
answers
53
views
How to optimize special array_intersect in hive sql executed by spark engine?
buckets is a column of type array<string>. The logic is similar to array_intersect, except only the prefix of each string in buckets (before the first -) is compared. How can I optimize the ...
- 11
0
votes
0
answers
133
views
Flutter app stuck on launcher screen on some Android devices (Impeller / Firebase / AppLinks suspected)
I’m experiencing an issue where my Flutter app works fine on most devices, but on some Android devices and emulators, it gets stuck on the icon/logo screen during the first launch.
Details from my ...
0
votes
1
answer
115
views
Spark 3.3.1: decimal(38,10) shows as 0E-10 in DataFrame and inserts as NULL into Hive (Parquet / insertInto)
Environment:
Apache Spark: 3.3.1
Hive: 3.1.3 (metastore + Hive server)
Table storage format: PARQUET
Insertion method: dataframe.format("hive").insertInto("db.table")
Problem ...
0
votes
1
answer
175
views
Apache Hive Docker container: HiveServer2 fails to bind on port 10000 (Connection refused in Beeline
I am running Apache Hive 4.0.0 inside Docker on Ubuntu 22.04.
The container starts, but HiveServer2 never binds to the port.
When I try to connect with Beeline:
sudo docker exec -it hive4 beeline -u ...
0
votes
1
answer
251
views
Trino with Hive- Infer parquet schema
Background: I am running trino with Hive catalog that reads data from S3. Setup and querying through trino over these tables is working fine with table's metadata stored in hive metastore.
Question: I ...
- 75
2
votes
0
answers
51
views
Count(*) query returns empty when using Tez, but works with MapReduce
I have hadoop + hive setup using docker, however when I try to run count(*) on my table it gives me an empty return when using Tez and the correct one when using MapReduce, the table is an external ...
- 21